Talk Summary

☀️ Quick Takes

Is Clickbait?

1-Sentence-Summary

Favorite Quote from the Author

can an llm system agent go out and do automatic pen testing no it can't um not yet um at least in my mind but can we use llms and AI to do some cool stuff yeah absolutely

💨 tl;dr

AI is revolutionizing bug bounty hunting by enhancing various stages like Recon, Application Analysis, Exploitation, Reporting, and Tool Maintenance. Choosing the right AI model and effective prompting are key. Verify AI outputs to avoid errors. Tools like Fabric framework, Burp GPT, and specialized bots like 'subdomain ninja' and Arcanum cybersecurity bot are game-changers.

💡 Key Ideas

• AI integration enhances bug bounty hunting across stages: Recon, Application Analysis, Exploitation, Reporting, Tool Maintenance.
• Choosing the right AI model (OpenAI GPT, Anthropic, Llama 3) is crucial; understand each model's strengths.
• System prompting and retrieval augmented generation (RAG) are techniques for building AI helpers; system prompting is currently more effective.
• Effective AI bots need substantial context and use of lower temperature for technical data.
• Verify AI outputs to avoid hallucinations; bots have a cutoff date for their training data.
• Daniel Miser's Fabric framework simplifies AI model usage and prompt creation.
• Effective prompting, including detailed system prompts, significantly enhances AI capabilities.
• Techniques to improve bot performance include bonuses, keyword terms, and specific reply rules.
• Bots like JD Jonathan Dunn's 'subdomain ninja' enhance Recon by generating subdomain permutations.
• The Arcanum cybersecurity bot assists in offensive and defensive security, analyzing code for vulnerabilities.
• AI can help deduce and exploit CVEs based on vendor descriptions and generate XSS attack strings.
• AI augmentation, such as Bounty Please, improves bug bounty reporting.
• Bots provide defensive fixes alongside offensive information to aid developers.
• Burp GPT extension integrates LLM capabilities into Burp Suite for interactive queries.
• Security data with open AI tools risk leakage; localized solutions are safer.
• Tools like nuclei ninja create rapid templates for bug hunting.
• Nessus uses Nasl scripting, with bots aiding in script creation and sharing new exploits.
• Additional training and consultancy services are offered on red, blue, and purple teaming.

🎓 Lessons Learnt

• Understand your AI model's strengths and limitations - Choose the right AI model based on what it excels at, e.g., coding, and be aware of any restrictions, like safety constraints in offensive security.
• Sharpen your tools - Spend time maintaining and improving your tools for efficiency and effectiveness.
• Leverage AI in different bounty hunting stages - Utilize AI in various stages (Recon, application analysis, exploitation, reporting) to enhance methodology and results.
• Stay updated with benchmarking information - Regularly check benchmarking articles, YouTube influencers, and white papers to make informed choices on AI models.
• Be cautious with AI model outputs - Always verify AI outputs to avoid inaccuracies.
• Choose the right AI model for your needs - Different AI models excel at different tasks (e.g., OpenAI for coding, LLaMA for in-house systems).
• Evaluate the use of agents vs system prompting - Decide between retrieval-augmented generation (RAG) for smarter AI or system prompting for context-rich interactions.
• Adjust temperature settings for AI creativity - Control AI’s creativity by setting the temperature; lower for literal outputs, higher for creative ones.
• Provide context for smarter bots - Feed your bots with ample context to improve their performance.
• Enable internet search for updated info - Allow bots to perform internet searches to access up-to-date information.
• Use the Fabric framework for AI prompts - Fabric helps bootstrap AI models with effective prompts for easy use.
• Great system prompts are crucial - Effective system prompts significantly enhance AI capabilities.
• Use structured markdown for prompting - Improve clarity and functionality in bot prompts with structured markdown.
• Specify bot expertise and sources - Define the bot's role and primary information sources to improve performance.
• Include detailed behavioral instructions - Instructions like 'enter a state of flow' can enhance bot performance.
• Bots perform better with bonuses - Giving bonuses and instructing bots to enter certain states improves their performance.
• Implement unconventional methods for better bot performance - Unconventional instructions in the bot's setup can enhance efficiency.
• Double-check bot outputs before using them - Always review bot outputs to ensure accuracy and reliability.
• Use bots for advanced subdomain discovery - Bots can enhance subdomain discovery by generating permutations of already found subdomains.
• Trust but verify when using multiple data sources - Cross-check bot outputs with traditional sources like Crunchbase.
• Use specialized bots for web analysis and security tasks - Bots like Arcanum Cyber Security Bot are useful for offensive and defensive security tasks.
• Automate code analysis tasks with AI - Use AI to analyze JavaScript code, parse API calls, identify vulnerabilities, and create documentation.
• Understand context limits of Bots - Be mindful of the memory limit for text bots can process in a single session.
• Use statistical data to identify vulnerabilities - Incorporate statistical data to pinpoint likely areas for certain attacks.
• Larger context windows enhance bot capability - More comprehensive projects can be input as context windows for bots increase.
• AI for pen testing is not fully reliable yet - Current AI systems require significant human input for pen testing.
• Create specialized bots for specific tasks - Build bots like the XSS mutation engine to save time and enhance efficiency.
• AI can deduce undisclosed CVEs - Analyze vendor descriptions of undisclosed CVEs to hypothesize potential vulnerabilities.
• Bypass weak filters with AI insights - AI can suggest methods to bypass weak filters for XSS and markdown injections.
• Enhance report quality and speed with AI - AI bots can automate and enhance bug bounty reporting for faster disclosure.
• Provide defensive fixes along with offensive information - Developers appreciate suggested defensive fixes alongside offensive findings.
• Use curl requests for bug reproduction - Developers prefer curl requests for reproducing bugs over security tools like Burp Suite.
• Incorporate Dom purify for input sanitization - Dom purify is a good boilerplate suggestion for sanitizing input in web applications.
• Use Burp GPT for integrated AI assistance - The Burp GPT extension enhances efficiency by allowing questions and traffic analysis within Burp Suite.
• Be cautious with security data in open AI tools - Ensure security data is not exposed when using open AI tools.
• Utilize bots for quick vulnerability checks - Tools like nuclei ninja automate vulnerability template creation, speeding up custom checks.
• Leverage chat channels for quick vulnerability response - Use chat channels to share new exploits and decide on immediate actions.
• Develop custom Nessus scripts for rapid scanning - Create custom Nessus scripts for quick vulnerability assessments while waiting for official checks.
• Utilize AI bots for script generation - Automate the creation of Nessus scripts or other security scanning scripts with AI bots.

🌚 Conclusion

AI integration in bug bounty hunting boosts efficiency and effectiveness across multiple stages. Selecting the right AI model and using effective prompts are crucial. Always verify AI outputs to avoid inaccuracies. Tools and frameworks like Fabric, Burp GPT, and specialized bots significantly enhance capabilities and streamline processes.

In-Depth

All Key Ideas

Jason Haddix's Talk on AI for Bounty Hunters

• Jason Haddix is discussing red AI specifically for Bounty Hunters.
• The talk will cover how to use AI to enhance bug bounty hunting.
• Jason has extensive experience in offensive security, including red teaming, pen testing, and bug bounty.
• The Bounty methodology can be broken down into stages: Recon, Application Analysis, Exploitation, Reporting, and Tool Maintenance.
• AI can be integrated into each stage of the Bounty methodology.
• Choosing the right AI model is crucial, as different models have different strengths.
• GPT models from OpenAI will be prominently used in the talk for demonstration purposes.
• Anthropics models are good for coding but have high safety restrictions that may limit their use in offensive security.
• Understanding what each AI model excels at is important for selecting the appropriate tool for the task.

Key Points on AI Models and Techniques

• Understanding the strengths of different AI models like OpenAI, Anthropic, and Llama 3 is crucial.
• Choices must be made between using retrieval augmented generation (RAG) or system prompting for building AI helpers.
• Agents, defined as small minibots for specific tasks, are popular but not yet as effective as system prompting.
• The concept of temperature in AI bots controls their level of creativity, with lower temperatures being more literal and higher temperatures being more creative.
• Effective AI bots require substantial context to function well.
• OpenAI GPT-4 is used for the examples in the talk, and lower temperatures are suggested for technical data.
• Users must verify data from AI bots because they can still produce hallucinations.

AI and Prompting Insights

• Bots have a cutoff date for the data they were trained on, limiting their knowledge to that point unless they can perform internet searches.
• Fabric is a framework created by Daniel Miser for using any AI model, simplifying prompt creation and AI usability.
• Effective prompting can significantly enhance AI capabilities, especially with well-written system prompts.
• System prompts set the stage for AI interactions and can be detailed with specific instructions and sources.
• There's debate on whether telling the bot it's an expert makes a difference, but some find it effective.

Bot Performance Improvement Techniques

• Giving bonuses to bots and instructing them to enter specific states improves their performance, based on AB testing by academics
• Using SEO-like keyword terms in bot instructions can enhance the bot's effectiveness
• Including related research terms and persona instructions significantly improves bot performance, as shown by AB testing
• Specifying rules for bot replies, such as using bulleted sentences, linking sources, and being thorough, is important
• The structure of a bot system prompt is crucial for building effective bots
• Applying bot prompting techniques to bounty hunting problems, like Recon, can be highly beneficial
• JD Jonathan Dunn's 'subdomain ninja' bot generates permutations of subdomains for thorough subdomain discovery

AI and Cybersecurity Bots

• AI can be used to assist in the Acquisitions phase by comparing outputs from different bots and databases like Crunchbase.
• Trust but verify the information provided by AI tools, as they may have discrepancies.
• The Arcanum cybersecurity bot, originally called SE GPT, is designed for both offensive and defensive security conversations.
• The Arcanum bot has been developed over a year and a half and is available on the GPD store.
• The Arcanum bot can analyze JavaScript and HTML code, providing insights into frameworks, libraries, technologies, API calls, authentication methods, and potential vulnerabilities.

Key Points about Bots and Vulnerabilities

• Bots have a context window limiting memory and text capacity in a session
• The Arcanum bot provides valuable information for discovery and exploitation phases
• Future versions of the bot aim to statistically identify areas prone to vulnerabilities
• Researchers found GPT-4 could exploit vulnerabilities with 40% success using known attack strings
• Current LLM systems cannot fully automate pen testing
• The xss mutation engine bot generates attack strings to bypass weak cross-site scripting filters

AI in Cybersecurity

• Using AI to deduce and exploit undisclosed CVEs based on vendor-reported descriptions
• AI-generated examples of vulnerable scenarios and bypass methods for XSS in markdown
• The development of an undisclosed CVE bot to inspire new approaches in vulnerability finding
• AI augmentation of bug bounty reporting to streamline and enhance submissions
• The creation of a reporting bot, Bounty Please, to quickly document and expand on vulnerability reports

Developer Tools and Practices

• Providing defensive suggested fixes along with offensive information can positively influence developers' responses
• Using tools like curl requests and Dom purify code helps developers reproduce and fix bugs more efficiently
• Customizing and verifying system prompts for bots is essential to get accurate and useful outputs
• Burp GPT extension integrates LLM capabilities into Burp Suite, allowing for interactive queries within the tool
• Security data shared with open AI tools pose a potential data leakage risk unless localized AI solutions are used
• Automation tools like nuclei ninja can rapidly generate useful templates for bug hunting and vulnerability checks

Nessus and Security Practices

• It's trained on all of the nuclei documentation
• Nessus has its own templating language called Nasl
• A bot was built to create Nasl scripts
• A chat channel was used to share new exploits among developers and security people
• Nessus checks can take several days to be released
• The speaker offers additional training on red, blue, and purple use cases
• The speaker's consultancy provides red teaming and penetration testing services

All Lessons Learnt

AI Model Utilization Tips

• Understand your AI model's strengths and limitations - Choose the right AI model based on what it excels at, e.g., coding, and be aware of any restrictions, like safety constraints in offensive security.
• Sharpen your tools - Spend time maintaining and improving your tools, akin to sharpening an axe before chopping a tree. This ensures efficiency and effectiveness in your tasks.
• Leverage AI in different bounty hunting stages - Utilize AI in various stages of bounty hunting (Recon, application analysis, exploitation, reporting) to enhance your methodology and results.
• Stay updated with benchmarking information - Regularly check benchmarking articles, YouTube influencers, and white papers to understand how different AI models perform and make informed choices.

Best Practices for Using AI Models

• Be cautious with AI model outputs: AI models can sometimes produce inaccurate results, so always verify their outputs.
• Understand model strengths: Different AI models excel at different tasks, so choose the right one for your needs (e.g., OpenAI for general coding, LLaMA for in-house systems).
• Choose between RAG and system prompting: Decide whether to use retrieval-augmented generation (RAG) for smarter AI or system prompting for context-rich interactions.
• Evaluate the use of agents: While agents (mini-bots) are popular, they may not be as effective as system prompting for now.
• Adjust temperature settings: Control the AI’s creativity by setting the temperature; lower settings for literal outputs, higher for creative ones.
• Provide context for smarter bots: Feed your bots with ample context to improve their performance.
• Use GPT-4 for practical examples: For practical purposes, leveraging GPT-4 from the OpenAI store can be highly effective.
• Keep temperature low for technical data: For technical applications, a lower temperature setting usually yields better results.
• Verify AI outputs: Always double-check AI-generated data to mitigate the risk of hallucinations and ensure accuracy.

Key Points for Enhancing AI Bots

• Bots have data cut-off dates: Bots like GPT-3 and GPT-4 have cut-off dates for their training data, meaning they don't have information beyond that unless they can perform internet searches.
• Enable internet search for updated info: To access up-to-date information, enable internet search in bots, especially those built in the GPT store.
• Use the Fabric framework for AI prompts: Fabric, created by Daniel Miser, is a useful framework to bootstrap AI models with effective prompts, making them easy to use on the command line.
• Great system prompts are crucial: Writing effective system prompts can significantly enhance AI capabilities. This is especially true as context windows expand.
• Structured markdown for prompting: Utilizing structured markdown to separate sections in prompts can improve clarity and functionality in bots.
• Specify bot expertise and sources: Clearly define the bot's role and primary sources of information to improve its performance, as done with business intelligence and acquisition data.
• Instructions for bot behavior matter: Including detailed behavioral instructions, even if unconventional (e.g., 'enter a state of flow'), can potentially enhance bot performance, though opinions on efficacy may vary.

Tips for Enhancing Bot Performance

• Bots perform better with bonuses: Giving bonuses to Bots and instructing them to enter certain states improves their performance, as proven by AB tests conducted by academics.
• Use weird machine tricks for better bot performance: Implementing unconventional methods, such as specific instructions in the bot's setup, can enhance bot efficiency.
• Format responses with detailed instructions: When dealing with a company's acquisitions, format each acquisition per line with the date and source to ensure clarity and focus on up-to-date responses.
• Include related research terms in bot instructions: Using SEO-like keyword terms related to the bot's task in the instructions can significantly improve bot performance. This has been tested and found effective.
• Set clear rules for bot replies: Instructions should include how the bot should format its replies, such as using bulleted sentences, linking sources, and being thorough. Repeating these rules can ensure consistency.
• Double-check bot outputs before using them: Always review the bot's outputs to ensure accuracy and reliability before utilizing them.
• Use bots for advanced subdomain discovery: Bots like Subdomain Ninja can enhance subdomain discovery by generating permutations of already found subdomains, similar to permutation scanning tools.

Best Practices for Using Bots and AI in Security

• Trust but verify when using multiple data sources - Using bots alongside traditional sources like Crunchbase can yield different results, so always cross-check the information.
• Use specialized bots for web analysis and security tasks - Bots like Arcanum Cyber Security Bot are useful for both offensive and defensive security tasks, providing tailored advice and automating parts of the recon process.
• It's okay to ask basic questions using AI tools - In a complex field like bounty hunting, it's impossible to know everything, so use AI bots to ask simple or clarifying questions without feeling self-conscious.
• Automate code analysis tasks with AI - Use AI to analyze JavaScript code, parse API calls, identify authentication methods, create documentation, and find potential vulnerabilities like XSS.

Key Points on Bot Capabilities and Vulnerabilities

• Understand context limits of Bots: Bots have a memory limit for how much text they can process in a single session, so be mindful when pasting large amounts of JavaScript to avoid exceeding this limit.
• Use statistical data to identify vulnerabilities: Incorporate statistical data on vulnerabilities to pinpoint likely areas for certain types of attacks, which can streamline the hacking process.
• Larger context windows enhance bot capability: As the context window for bots increases, you can input more comprehensive projects, improving the bot's functionality and output.
• Exploiting vulnerabilities with GPT-4: While GPT-4 can exploit vulnerabilities with enough context, its success rate is relatively low compared to traditional methods, indicating it's still a work in progress.
• AI for pen testing is not fully reliable yet: Current AI systems, including LLM agents, are not yet capable of fully automatic pen testing without significant human input.
• Create specialized bots for specific tasks: Building bots like the XSS mutation engine, which automates the process of bypassing weak cross-site scripting filters, can save time and enhance efficiency in bug bounty hunting.

AI in Cybersecurity

• Using AI to deduce undisclosed CVEs: AI can be leveraged to analyze vendor descriptions of undisclosed CVEs to hypothesize potential vulnerabilities and exploit methods.
• Bypassing weak filters with AI insights: AI can suggest methods to bypass weak filters, such as breaking up image tags or using data URLs for XSS and markdown injections.
• Enhancing report quality and speed with AI: AI bots can automate and enhance bug bounty reporting, making it faster and more detailed, which helps in the race to disclose vulnerabilities efficiently.

Best Practices for Developers and Security Tools

• Provide defensive fixes along with offensive information: Developers are more responsive and appreciative when they receive suggested defensive fixes alongside the offensive findings.
• Use curl requests for bug reproduction: Developers prefer curl requests for reproducing bugs as they often lack security tools like Burp Suite.
• Incorporate Dom purify for input sanitization: Dom purify can be a good boilerplate suggestion for sanitizing input in web applications.
• AB test and verify bot outputs: Continuously tweak and test your bot prompts to ensure accurate and useful outputs, as they may not work perfectly right away.
• Use Burp GPT for integrated AI assistance: The Burp GPT extension allows you to ask questions and analyze traffic directly within Burp Suite, enhancing efficiency.
• Be cautious with security data in open AI tools: Using open AI tools like Burp GPT means your security data may be exposed unless the tool can operate locally.
• Leverage bots for quick vulnerability checks: Tools like nuclei ninja can automate the creation of vulnerability templates, speeding up the process of writing custom checks.
• Utilize bots for specific bug hunting tasks: Bots can be used to write scripts for tasks like subdomain takeover checks or creating Shodan queries, simplifying complex bug hunting activities.

Strategies for Rapid Vulnerability Response

• Leverage chat channels for quick vulnerability response: Use chat channels with developers and security teams to share new exploits and decide on immediate actions outside of normal policies.
• Develop custom Nessus scripts for rapid scanning: When waiting for official checks from Tenable, create custom Nessus scripts using their templating language, Nazle, to quickly scan for new vulnerabilities.
• Utilize AI bots for script generation: Build or use AI bots to automate the creation of Nessus scripts or other security scanning scripts to expedite vulnerability assessments.