In this episode I discuss the potential negatives of using AI Coding Assistants in software development. I cover topics such as:
Or listen at:
Published: Wed, 12 Jul 2023 15:00:24 GMT
Transcript
Hello and welcome back to the Better ROI from Software Development Podcast.
Over the last couple of weeks, I've been looking into the idea of AI Coding Assistants, the use of large language models like ChatGPT to assist software developers. In episode one 70, I introduced the idea of the AI Coding Assistant, and in last week's episode, 171, I looked at the expected benefits.
In this episode, I wanted to look at the potential negatives. Specifically I wanted to cover: * Overreliance and skill degradation * Lack of creativity and innovation * False positives and incorrect suggestions * Limitations with domain-specific knowledg * Dependency on internet connectivity * Adoption and learning curve * Privacy and security concerns
Let's take each of these in turn:
Overreliance and skill degradation.
Relying too heavily on AI Coding Assistants may hinder developers from developing their problem solving and coding skills. It is important to strike a balance between using the assistant as a tool and actively engaging in the coding process.
We've seen examples of master craftspeople effectively aging out of industries where automation and cheap production has taken hold. Many master craftspeople find the skills are no longer needed and there isn't the demand to pass them down to the next generation, ultimately leading to the skills dying out.
An alternative way to think about this is, why learn how to change the tire of a car if you only ever use Uber?
In my generation, it would've been unthinkable not to learn how to drive and carry out basic car maintenance from around the age of 17.
Now, it's possible, and possibly even desirable, to never learn either to drive or to carry out car maintenance - which is all fine until you need to do one of those things.
If we over-rely on AI Coding Assistants and lose those skills, how will we repair our metaphorical puncture in our AI generated code?
And losing skills also means that we're no longer able to question the output of the AI Coding Assistant. Lacking the experience and capability to review the AI generated code means putting pretty much a hundred percent trust into it, which at this stage seems highly risky.
Lack of creativity and innovation.
AI Coding Assistants primarily work based on existing patterns and code examples. They do not possess the same creativity and innovation thinking that human beings bring to the table. Developers might miss out on exploring novel approaches or unique solutions if they solely rely on AI suggestions.
In episode 169, I talked about this very thing, the diminishing of original content and thought.
Now this may seem like an academic concern, but with learning communities such as software development, I can see this being a real and impending risk. Advancement in software development relies heavily on disparity of thought, new ideas to address old problems, and then being able to receive kudos for your contribution to the overall practice.
This slowing of creativity and innovation would ultimately lead to the over reliance and skill degradation.
False positives and incorrect suggestions.
AI Coding Assistants may occasionally provide incorrect or misleading suggestions. While they strive to be accurate, they're limited by their data they have been trained on. Developers need to carefully review and validate the suggestions to avoid introducing bugs or errors into their code.
And this is why I talked earlier about putting a hundred percent trust in AI generated content as being highly risky.
Now, we wouldn't put a hundred percent trust in a single developer. We will be putting systems in place around that developer, such as unit testing, peer review, small batch sizes to make sure our developer's work is effective, and secure.
Our AI Coding Assistant is ultimately no different than our developer.
We need to ensure the relevant system is in place to provide guardrails.
A hundred percent trust is not an option. It isn't recommended, suggested, or advised.
Limitations with domain specific knowledge.
AI Coding Assistant might not possess deep knowledge of specific domains or industries. They might struggle to understand complex business logic or domain specific requirements, potentially leading to suboptimal or incorrect suggestions.
The AI Coding Assistant will be based on large language models, which are built using publicly accessible software code, whereas every organization will have their own business logic - otherwise, why are you writing software - it's that organization specific business logic, which can provide the game changer in any industry.
By definition, the AI Coding Assistant will be very biased to a generic approach, not one tailored to your specific organizational needs. It has been built to help as many organizations as possible, and because of this, we may find that much of our software code is very consistent with the masses, but will present limitations for the organizational specific requirements that we have.
Dependency on internet connectivity.
Many AI Coding Assistants require a stable internet connection to function properly. In situations where internet access is limited or unreliable, developers may face disruption or be unable to utilize the assistance features fully.
I'm worried this may produce a new level of privilege within software developers, those with reliable, consistent internet connections and those without - and new "haves" and "have nots".
And it may not even be along the traditional wealth lines. For example, Italy has banned a use of chatGPT over the fears of data security and the impact of jobs - and it is rumored that Germany may follow the same approach. However, by doing this, does Italy put its software developers in a worse state compared to other countries? Are they going to miss out on the skills needed for the next decade of software development?
Adoption and learning curve.
Introducing an AI Coding Assistant to a development team requires time for learning and adaption. Developers need to familiarize themselves with the tools functionality, integrate it into their workflows, and adjust their coding practices accordingly. This initial learning curve can temporarily impact productivity.
While AI Coding Assistants can seem like magic and potentially provide great benefits, there will need to be that learning and adoption curve, and this is not necessarily going to be welcomed with open arms.
For example, I've previously trialed AWS Code Whisperer and personally found it counterproductive and plain odd. Now, I put my hands up to that being a very drive-by testing and not sticking with it. It didn't work for me the way that I worked within the first few hours, so I removed it.
Now, that may seem like an unfair test, but it is no different than the approach many developers will take when first assessing an AI Coding Assistan - we're a busy lot, and if it's getting in the way, then it would quickly be sidestepped.
Plus, of course, there is also the fear of AI taking our software development jobs - all industries rebel against automation.
Privacy and security concerns.
Using AI Coding Assitants often involves granting them access to the code base, which may raise privacy and security concerns, especially when dealing with proprietary or sensitive information. It's crucial to carefully evaluate the security measures and data handling policies of the AI coding assistant provider.
I see two key areas. Firstly, the danger of exposing proprietary information and secondly, AI Coding Assistants becoming a new method of cyber attack.
I talked about the risk of exposing proprietary information back in episode 169 - Samsung has repeatedly fallen foul of this already with their private information being first shared with chat GPT, and then via chatGPT exposed to others. This has led many organizations to simply ban the use of AI Coding Assistants and relevant tools - but that could only really be a short term measure. It won't be a practical response for very long.
And on the security side, cyber criminals will attempt to exploit any new technology. The AI Coding Assistant represents a potential gold mine to an attacker.
Imagine if an attacker can an influence your AI Coding Assistant to slip in subtle security vulnerabilities into your production system or simply co-opt your AI Coding Assistant into using its trusted status to do things on their behalf.
The AI assistant field is very young, and as such will not have received much scrutiny in potential security problems, but it is a safe bet to assume that cyber criminals are already looking at ways to do this.
And until the concerns over privacy and security are addressed, we can expect organization to implement blanket bans on the use of AI Coding Assistant, which will impact adoption again, leading to those "haves" and "have nots".
My current client, for example, has one such ban. This effectively limits my ability to learn and adopt AI Coding Assistants - I can only use them for work outside of that client - thus, it's difficult to build up my experience and muscle memory. And while I suspect that my current client will lift this ban soon, it is likely to be ahead of the pack with many industry developers simply being unable to use AI Coding Assistants - and I can honestly see a time that this is going to be such a detriment to the software developer that it'll be part of the reasons why we leave jobs to find organizations that allow us to use the tools that we feel make us most productive and to avoid being left behind in the marketplace.
In this week's episode, I wanted to go through some of the potential negatives for AI coding assistance. I talked about: * Overreliance and skill degradation * Lack of creativity and innovation * False positives and incorrect suggestions * Limitations with domain-specific knowledge * Dependency on internet connectivity * Adoption and learning curve * Privacy and security concerns
In next week's episode, I want to take a look at the future of AI Coding Assistants.
Thank you for taking the time to listen to this podcast. I look forward to speaking to you again next week.
ChatGPT transcript
This episode used content generated through conversation with ChatGPT. The below is the full conversation with ChatGPT.
User:
List the advantages to a software developer using an ai coding assistant
ChatGPT:
Using an AI coding assistant can offer several advantages to software developers:
Increased productivity: An AI coding assistant can help automate repetitive tasks, provide code suggestions, and offer quick fixes for common errors. This accelerates development workflows and allows developers to write code more efficiently.
Code optimization: AI coding assistants often analyze code and provide suggestions for optimization. They can identify areas where code can be refactored or improved, leading to cleaner and more efficient implementations.
Learning and knowledge enhancement: AI coding assistants can provide real-time feedback, helping developers learn best practices and coding conventions. They can also offer insights into new technologies, libraries, and frameworks, keeping developers up to date with the latest advancements in the field.
Error detection and debugging: AI coding assistants can assist in identifying and fixing errors in code. By analyzing the codebase, they can catch potential bugs and offer suggestions to resolve them, saving developers time and effort in the debugging process.
Collaboration and code consistency: AI coding assistants can help maintain code consistency across a team by enforcing coding standards and style guidelines. They can also aid in collaborative coding, suggesting changes or improvements to code written by other team members.
Code documentation: AI coding assistants can generate documentation based on code analysis, making it easier for developers to document their code and keep it up to date. This can be particularly helpful when working on complex projects with multiple contributors.
Code completion and suggestions: AI coding assistants offer intelligent code completion, automatically suggesting code snippets, function signatures, and variable names based on context. This speeds up development and reduces the chances of introducing syntax errors.
Overall, using an AI coding assistant can enhance a developer's productivity, code quality, and learning experience, leading to faster development cycles and improved software outcomes.
User:
What negatives could be felt for using ai coding assitants
ChatGPT:
While AI coding assistants provide numerous benefits, there are a few potential negatives to consider:
Overreliance and skill degradation: Relying too heavily on AI coding assistants may hinder developers from developing their problem-solving and coding skills. It's important to strike a balance between using the assistant as a tool and actively engaging in the coding process.
Lack of creativity and innovation: AI coding assistants primarily work based on existing patterns and code examples. They might not possess the creativity and innovative thinking that humans bring to the table. Developers might miss out on exploring novel approaches or unique solutions if they solely rely on AI suggestions.
False positives and incorrect suggestions: AI coding assistants may occasionally provide incorrect or misleading suggestions. While they strive to be accurate, they are limited by the data they have been trained on. Developers need to carefully review and validate the suggestions to avoid introducing bugs or errors into their code.
Limitations with domain-specific knowledge: AI coding assistants might not possess deep knowledge of specific domains or industries. They might struggle to understand complex business logic or domain-specific requirements, potentially leading to suboptimal or incorrect suggestions.
Privacy and security concerns: Using AI coding assistants often involves granting them access to the codebase, which may raise privacy and security concerns, especially when dealing with proprietary or sensitive information. It's crucial to carefully evaluate the security measures and data handling policies of the AI coding assistant provider.
Dependency on internet connectivity: Many AI coding assistants require a stable internet connection to function properly. In situations where internet access is limited or unreliable, developers may face disruptions or be unable to utilize the assistant's features fully.
Adoption and learning curve: Introducing an AI coding assistant to a development team requires time for learning and adaptation. Developers need to familiarize themselves with the tool's functionalities, integrate it into their workflow, and adjust their coding practices accordingly. This initial learning curve can temporarily impact productivity.
By understanding these potential negatives and taking appropriate measures, developers can make the most of AI coding assistants while mitigating any drawbacks. It's essential to strike a balance and leverage the assistant's capabilities effectively within the development process.
User:
What is the future of ai coding assitants
ChatGPT:
The future of AI coding assistants holds great potential for further advancements and improvements. Here are a few aspects that may shape their development:
Enhanced contextual understanding: AI coding assistants will continue to evolve in their ability to understand code context, project requirements, and user intent. This will enable them to provide even more accurate and relevant suggestions, making them increasingly indispensable tools for developers.
Integration with more development tools: AI coding assistants will likely integrate seamlessly with various development environments, IDEs, and code editors, offering real-time assistance within the developer's preferred workflow. This integration will streamline development processes and make AI coding assistants more accessible to a broader range of developers.
Customization and adaptability: Future AI coding assistants may allow developers to customize their behavior and preferences according to their coding style, project requirements, and team conventions. This flexibility will enable developers to tailor the assistant's suggestions to their specific needs, enhancing productivity and code quality.
Deeper understanding of codebase and project context: AI coding assistants may become more proficient in understanding and analyzing entire codebases, enabling them to provide comprehensive suggestions and improvements beyond individual code snippets. They may offer insights into architectural design, code organization, and potential optimizations at a system level.
Collaboration and knowledge sharing: AI coding assistants could facilitate collaborative coding environments by providing real-time suggestions, conflict resolution, and code merging assistance. They may also serve as repositories of collective knowledge, storing and sharing best practices, patterns, and reusable code components across development teams.
Natural language interfaces: AI coding assistants might incorporate natural language processing capabilities, allowing developers to interact with them using more human-like language and commands. This could make the interaction more intuitive and reduce the learning curve for using the assistants effectively.
Smarter debugging and error prevention: Future AI coding assistants may become more adept at identifying and preventing errors at an early stage. They could offer sophisticated debugging support, intelligent error detection, and suggest fixes based on comprehensive code analysis, minimizing the time spent on debugging.
Overall, the future of AI coding assistants looks promising, with the potential to revolutionize software development by increasing productivity, improving code quality, and fostering collaboration among developers. As the technology progresses, developers can expect more powerful and intelligent coding assistants to assist them in their daily work.
User:
Are ai coding assistants likely to support organisational process, concepts, styles, etc
ChatGPT:
Yes, AI coding assistants are likely to support organizational processes, concepts, styles, and other guidelines. As these assistants evolve, they can be designed to align with specific organizational coding standards, style guides, and development processes.
Here's how AI coding assistants can support organizational aspects:
Coding conventions and style: AI coding assistants can be trained to understand and enforce specific coding conventions and style guidelines followed by an organization. They can provide suggestions and corrections to ensure code consistency across projects and teams.
Project-specific requirements: Organizations often have project-specific requirements or domain-specific conventions. AI coding assistants can be configured to understand and adapt to these requirements, providing suggestions and validations that align with the organization's needs.
Design patterns and architectural principles: AI coding assistants can assist developers in implementing design patterns and adhering to architectural principles. They can recognize patterns in code and suggest appropriate design patterns or architectural improvements based on the organization's preferred approaches.
Quality assurance and code reviews: AI coding assistants can aid in code reviews by automatically detecting potential issues, security vulnerabilities, or performance bottlenecks. They can assess code against organizational quality standards and provide feedback to ensure code quality and adherence to best practices.
Integration with existing tools and workflows: AI coding assistants can be integrated with existing development tools and workflows, ensuring seamless adoption within an organization's software development processes. They can provide suggestions and feedback directly within IDEs, code editors, or version control systems, enabling developers to work efficiently.
Knowledge sharing and onboarding: AI coding assistants can contribute to knowledge sharing within an organization by storing and retrieving code snippets, best practices, and project-specific insights. They can assist in onboarding new developers by providing guidance and context-specific suggestions.
By incorporating organizational processes, concepts, styles, and guidelines into AI coding assistants, organizations can ensure that these tools align with their unique requirements and contribute to consistent and efficient software development practices.
User:
Explain more about the onboarding benefits
ChatGPT:
Certainly! AI coding assistants can offer several benefits during the onboarding process of new developers in an organization:
Guided learning: AI coding assistants can provide contextual guidance to new developers, helping them understand the organization's coding conventions, best practices, and style guidelines. They can offer real-time suggestions and explanations, aiding in the learning process and reducing the time required for new developers to become familiar with the coding standards.
Code examples and snippets: AI coding assistants can serve as repositories of code examples and snippets specific to the organization's projects and domain. New developers can leverage these examples to understand common patterns, implementations, and approaches followed by the organization. This accelerates their learning and enables them to ramp up quickly.
Error prevention and feedback: AI coding assistants can proactively detect potential coding errors, security vulnerabilities, or performance issues in the code written by new developers. By offering real-time feedback and suggestions for improvement, the assistants help new developers avoid common pitfalls and ensure their code adheres to the organization's quality standards.
Consistency and adherence to standards: AI coding assistants can enforce coding standards and style guidelines during the onboarding process. They can automatically flag deviations and provide suggestions to align code written by new developers with the organization's established practices. This ensures consistency across the codebase and promotes adherence to the organization's coding standards from the start.
Efficient problem-solving: AI coding assistants can assist new developers in problem-solving by offering suggestions and insights based on their understanding of the codebase and existing patterns. This can help new developers navigate through complex codebases and identify potential solutions more efficiently.
Continuous learning and improvement: AI coding assistants can learn from the coding practices and patterns of experienced developers in the organization. As new developers interact with the assistants, their feedback and interactions contribute to improving the assistants' capabilities over time, creating a cycle of continuous learning and improvement.
By leveraging AI coding assistants during the onboarding process, organizations can facilitate the integration of new developers into their development teams more effectively. These assistants provide valuable guidance, promote adherence to coding standards, and foster a culture of learning and collaboration. Ultimately, this leads to faster onboarding, improved code quality, and enhanced productivity within the organization.
User:
Do we expect ai coding assistants to be an essential tool for any software developer? If not not yet, when?
ChatGPT:
While AI coding assistants offer significant benefits, it's not guaranteed that they will become an essential tool for every software developer in the immediate future. The adoption and necessity of AI coding assistants can vary depending on factors such as the developer's preferences, project complexity, development environment, and available resources.
However, as AI coding assistants continue to evolve and improve, there is a possibility that they will become increasingly essential tools for software developers. The advancements in natural language processing, code understanding, and machine learning techniques will enhance the capabilities of AI coding assistants, making them more valuable in various scenarios.
The point at which AI coding assistants become indispensable tools for all software developers will depend on several factors, including:
Advancements in AI technology: Continued advancements in AI, particularly in natural language processing, code analysis, and context understanding, will contribute to the effectiveness and reliability of AI coding assistants. As these technologies mature, the assistants will become more capable and reliable in assisting developers across different programming languages and development environments.
Integration and compatibility: Seamless integration of AI coding assistants into popular development environments, IDEs, and code editors will play a crucial role in their widespread adoption. When developers can easily incorporate the assistants into their existing workflows without significant disruptions, the barrier to entry will be lowered, making them more accessible and likely to be adopted widely.
Trust and reliability: Developers need to trust the suggestions and assistance provided by AI coding assistants. As the assistants demonstrate higher accuracy, better understanding of code context, and reduced false positives, developers will rely on them more confidently. Building trust will be essential for wider acceptance and adoption.
Industry support and best practices: Widespread adoption of AI coding assistants may also depend on the support and endorsement of influential organizations, industry leaders, and established coding standards bodies. When these entities recognize the value and benefits of AI coding assistants and endorse their use, it can accelerate their adoption across the software development community.
While it is challenging to predict the exact timeframe, it is conceivable that AI coding assistants could become an essential tool for many software developers within the next several years. As the technology continues to advance and address current limitations, AI coding assistants have the potential to significantly enhance developer productivity, code quality, and collaboration in the software development process.
User:
Compare the ai coding experience to IDE auto completion like visual studio's intillisense
ChatGPT:
AI coding assistants and IDE auto-completion, such as Visual Studio's IntelliSense, serve similar purposes of assisting developers in their coding tasks. However, there are notable differences between the two:
Contextual Understanding: While IDE auto-completion like IntelliSense provides suggestions based on the current context within the code editor, AI coding assistants leverage machine learning algorithms to analyze a broader context. They consider patterns from vast codebases and take into account code semantics, coding conventions, and best practices to offer more comprehensive and intelligent suggestions.
Code Quality and Refactoring: AI coding assistants often go beyond simple code completion and offer suggestions for code refactoring, optimizations, and error prevention. They can identify potential code smells, performance bottlenecks, and security vulnerabilities, providing valuable feedback to improve code quality. IDE auto-completion typically focuses on providing suggestions based on existing code within the project.
Learning and Adaptability: AI coding assistants continuously learn from the code they analyze and interactions with developers. They can adapt to specific coding styles, project requirements, and evolving best practices. IDE auto-completion, while customizable to some extent, primarily relies on predefined code patterns and rules.
Integration with Development Workflow: IDE auto-completion tools like IntelliSense are tightly integrated into the IDE and provide suggestions within the coding environment. AI coding assistants, on the other hand, can be integrated with various development environments, IDEs, or code editors, extending their functionality beyond a specific tool and offering cross-platform support.
Language Support: IDE auto-completion tools are typically designed to support specific programming languages or frameworks within a specific IDE. AI coding assistants, depending on their design, can potentially offer support for multiple programming languages and frameworks, making them more versatile in multi-language or diverse development environments.
Autonomy and Assistance: AI coding assistants can provide more proactive assistance, offering suggestions and guidance as developers write code. They can detect potential issues and provide recommendations even before the developer initiates a request. IDE auto-completion, on the other hand, primarily responds to triggers or manual requests from the developer.
In summary, AI coding assistants provide a more advanced and intelligent coding experience compared to IDE auto-completion like IntelliSense. They leverage machine learning algorithms, analyze broader code contexts, offer suggestions beyond simple completion, and continuously learn and adapt. While IDE auto-completion is still a valuable and widely used feature, AI coding assistants have the potential to revolutionize the coding experience by enhancing productivity, code quality, and error prevention.
User:
Summarise IDE auto-completion like IntelliSense
ChatGPT:
IDE auto-completion, exemplified by IntelliSense in Visual Studio, is a feature that suggests code completions and provides context-aware suggestions as developers write code. It offers the following key characteristics:
Contextual suggestions: IntelliSense analyzes the code context, including variables, functions, and available libraries, to provide relevant suggestions as developers type. It assists in completing code statements, method signatures, and property names.
Quick documentation: IntelliSense often displays tooltips or brief documentation for suggested items, offering information on function parameters, return types, and available options. This helps developers understand the usage and purpose of code elements.
Customizable behavior: Developers can often customize IntelliSense preferences, including the order and type of suggestions, enabling them to tailor the auto-completion experience to their preferences and coding style.
Language and framework support: IntelliSense is typically designed to support specific programming languages and frameworks. It provides language-specific suggestions and understands language-specific constructs, making it more effective for code completion within those languages.
Trigger-based suggestions: IntelliSense suggestions are typically triggered by specific keystrokes or symbols, such as typing a period (".") to access member suggestions or pressing Ctrl+Space to invoke suggestions explicitly.
Integration within IDE: IntelliSense is tightly integrated into the IDE, providing suggestions directly within the code editor. It ensures a seamless coding experience without the need for external tools or plugins.
Overall, IDE auto-completion like IntelliSense enhances developer productivity by reducing manual typing and offering relevant suggestions in real-time. It provides contextual assistance, improves code accuracy, and helps developers explore available options and code elements within the integrated development environment.
User:
Summarise IDE auto-completion like IntelliSense for a non technical managerial audience
ChatGPT:
IDE auto-completion, such as IntelliSense in Visual Studio, is a feature that assists software developers as they write code. It provides suggestions and completes code statements based on the context of what they are typing. This helps developers write code faster and with fewer errors.
IntelliSense understands the programming language and frameworks being used and suggests appropriate code elements like functions, variables, and properties. It even provides quick documentation to help developers understand how to use these code elements correctly.
This feature is seamlessly integrated into the development environment, so developers don't need to use external tools. It improves the coding experience by reducing the need for manual typing and offering helpful suggestions that align with best practices.
In summary, IDE auto-completion like IntelliSense is a valuable tool that assists developers by suggesting and completing code as they write, ultimately making the coding process more efficient and less error-prone.