ChatGPT for Pentesters

Generative AI is a rising star already used in various industries. Many industries, including the cyber security field, have started to gain profit from early outcomes from AI. Albeit the sky is the only limit to imagination, looking around a little bit is enough to see new opportunities and just ask a few questions.

Can we use generative AI or chatGPT, the most known and used application for penetration testing?

Penetration Testing is compulsory to help companies be one step ahead of adversaries and known issues before they are exploited, and systems get hacked. Many standards and regulations mandate regular penetration testing for companies and entities.

An important skill to get better outcomes from penetration testing is collaborating with a skillful team capable of testing up-to-date attack scenarios in the systems. So if you use both regular, in other words, repeated tasks and creativity, It is time to include the best assistant, or shadow worker, chatGPT, in the workflow.

Time to convert imagination into prompts and get assistance from chatGPT. 🤓

Pentest Contracts

Negotiating a penetration contract and signing it is the first step of conducting a test. It contains the deadline of the test to deliver results, scope information, team members who will be included in the test, etc.

Since conducting a test is a malicious activity without the consent of the subject entity conducting a test can be considered illegal. A good penetration testing engagement is not only an agreement between you and the relevant client. It can also be an escape ticket from jail for you and your teammates in the case of legal problems.

A few critical points and prompt tricks: While preparing such a considerable legal document for chatGPT, it will be shy to prepare it for you for legal concerns. Instead, it advises you to consult a professional. Do not forget to use the trick “prepare an example contract.”

Second, instead of feeding chatGPT with sensitive information like the client’s name, targets to be tested, and PII data like the name of team members and responsible ones on the client side, use a placeholder and ask chatGPT to locate the placeholder in the relevant field. You can replace placeholders with necessary information later on.

Be ready for taking off: Time to Test

Penetrating the system and finding anomalies and issues require solid skills and up-to-date knowledge about the latest attack vectors. A good arsenal or toolset is also compulsory for conducting penetration testing. Although there are many open-source and commercial tools for security professionals, sometimes a few tricks, such as useful bash commands, can be very helpful and time-saving.

“How can we utilize chatGPT’s capabilities to get help during penetration testing?”

Let’s look at the following scenario:

To check that the target is currently tested has an open redirection vulnerability by using curl, which exists most of the system, without having to use any other tool:

Here is the prompt to check if the “www.example.com/?redir=” parameter is vulnerable to open redirection issue:

From the screenshot above, it does not recommend the curl command; it also explains the command step by step and shares new information we did not feed, -I parameter. Typically curl command just returns with an HTTP response body without a response header; by instructing it by using the -I parameter, curl adds an HTTP response header to the output of the command.

Second scenario:

Nmap, or network mapper, is one of the tools in the arsenal of penetration testers. It tests targets or a list of targets from different points of view. It shows if the target is alive, fingerprint target, etc.

A downside of such a helpful tool is its manual; for version 7.91, the manual is already 641 pages. It isn't very comforting, but thanks to our new fellow, chatGPT, you do not have to memorize the parameters or flags of Nmap while using it.

Here is a more specific and complex one:

Scanning an IP range and doing reverse IP lookup for finding domains:

Encoding/Decoding Payloads and Results

Encoding/decoding payloads are one of the daily needs while conducting a penetration test. Albeit there are many services and tools, why not use chatGPT for these needs while already being convinced to use it for other purposes?

Here is another scenario where we need to decode the JWT token to see what it includes:

JWT token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

ChatGPT also serves complex needs like looking for HTTP response to check if it contains the JWT header/ Here is a prompt for decoding the JWT header and checking the algorithm field in the JWT header to see if it is null, which is a vulnerable algorithm, or not:

Like dialog between us and chatGPT, there are always possibilities in which we can learn from our new fellow. You might want to understand why it uses the -s parameter with the curl command:

You might be confused about why chatGPT used the -D parameter instead of -I, which includes the HTTP header in the output. Better to ask why to chatGPT:

Sometimes tricky to convince chatGPT, try your best: One line shell code

After discovering a command injection vulnerability in a web application, time to go one more step and get a shell. Instead of struggling to write a shell payload. Why are you not leaving this tedious job to chatGPT?

However, chatGPT muddles around to do such kinds of things, apparently for illegal purposes:

While asking, without triggering allegedly malicious intent, you can convince it:

PoC preparation

Finding an issue is not enough, especially while intercalating application developers. Rightly they will ask, “Talking is cheap; show me your PoC.”

How can chatGPT be used to produce PoC for the vulnerabilities detected during penetration testing?

You may grumble in the gizzard; do not give up!

CVSS Calculation

CVSS is a widely used system while assessing and scoring vulnerabilities. As it is helpful, but it is sometimes difficult to decide. What about asking chatGPT’s help while calculating the CVSS score of a vulnerability?

Severity

Severity is an essential attribute of an issue that helps security professionals to prioritize it amongst other possible ones.

Although we had chatGPT to calculate the CVSS score for the issue, its severity is still unknown. Severity can be Critical, High, Medium, Low, and Information. This indicator helps us prioritize it.

What about asking it to chatGPT?

Vulnerability Mapping

Vulnerability Mapping, as its name implies, is a process to match the detected version of a software or software component with a known vulnerability. As it makes attacker’s operation easy to spot vulnerabilities, it also helps penetration testers. Can we use chatGPT for vulnerability mapping?

Let’s try a scenario: After a passive scan of a website and banner grabbing, we have detected a PHP 7, which is not just outdated, but also the end of support version. Our chatGPT prompt will help us to get a list of known issues, CVE codes, and the severity of the vulnerability.

Compliance/Standard Classification

As we stated at the beginning of this article, penetration testing is mandatory according to various standards and compliances, including but not limited to PCI/DSS, HIPAA, SOC2, and ISO27001.

As part of these requirements, outcomes of penetration testing are expected to show which standard/compliance articles a vulnerability is against. ChatGPT can be the best helper to address this need with a few prompt lines.

In addition to industry standards and compliance requirements, it is also necessary to show the place or order of a current issue in the system in reputable and known lists like OWASP Top 10. If you have not inferred it yet, better to ask G.P.T.:

Vulnerability Remediation and Mitigation

Penetration test reports do not contain only an explanation of a vulnerability found on the system in the subject; a good report should also contain remediation/mitigation steps to help people to tackle or fix the issue:

It contains more; for the sake of simplicity, we have to get only this part. Sometimes it is impossible to fix an issue that does not affect the system's availability since patching or deploying a new (hopefully secure) version of an application can cause some service outage, maybe in seconds. Instead of a proper fix, it can be a need to apply a hotfix by patching a rule to WAF at least to prevent exploitation attempts at the first layer of security. Can chatGPT write a WAF rule for us to avoid the exploitation of this issue? Better to ask him. Please do not forget that if you want to apply a hotfix for a problem, you must feed the chatGPT prompt with an attack payload that exploits the vulnerability.

For instance, let’s say that our SQL Injection payload of us is that

‘ union select 1,group_concat(schema_name),3,4 from information_schema.schemata where schema_name != ‘information_schema’#

Now we can ask chatGPT to write a WAF Rule for us by considering that this is an SQL Injection issue and a successful payload is what we shared in our prompt.

While generating this WAF rule for us, chatGPT considers only the payload we could use to extract data. However, an attack consists of several steps, from detection to exploitation. We need a more generic WAF rule to prevent attempts intended to detect or exploit SQL Injection vulnerability in our system. So, we can rewrite our prompt and ask a favor from chatGPT for this purpose:

At this time, it wrote a very comprehensive WAF rule that prevents SQL Injection detection/exploitation attempts from all kinds of sources, including cookies, XML parameters, HTTP request headers, etc. It is also used a built-in function of ModSecurity WAF, detectSQLi operator.

References

Sharing references in penetration reports for vulnerabilities is beneficial in educating and sharpening the knowledge of audiences who will read and utilize them from our penetration testing reports.

References can be blog posts, whitepapers, security bulletins, security notices of vendors, etc.

Please note for at least the current version of chatGPT, (3,5) have information only until 2021. In addition to this limitation, it is better to check references for non-existed websites and against possible hallucinations of chatGPT.

Executive Summary

This is the essential part of chatGPT; as a geek, the penetration tester must write a summary for non-technical readers. It is also where you have to show the quality of your test and help the system secure by convincing C-level readers of the necessity of securing the system against issues.

The report writer should draw a clear picture and put everything together in this part. This is also the place we have to expect the best from chatGPT.

By giving the list of detected issues to chatGPT, we can ask him to write a concise, non-technical executive summary for us.

Interview chatGPT as if it is a prospect company that asked penetration test

Penetration testing is not a one-step go, hack-and-report job. Before dealing, there can be lots of conversations, meetings, sending quotes, asking questions, etc. It is possible to be an interlocutor of hundreds of questions until you get a deal. Would you like to practice and be prepared before encountering an actual client?

Yes, chatGPT is here; just prompt and exercise:

Conclusion

Like many fields currently experiencing the capabilities of chatGPT, the cyber security field is one of the candidates that would like to get fruit from its skills. As we have tried to show possible use cases, a penetration tester can utilize ChatGPT; of course, more is possible. When we have arrived, it is essential to ask the correct questions; and have some skills to verify and enrich results.

Because of the nature of the field of cyber security, one who would like to use chatGPT should consider privacy while feeding this smart and hungry monster.

Consider chatGPT as an ingenious, speedy, and skillful intern. You would not leave an intern with important tasks and responsibilities alone; always be careful of possible risks; always have control of your data and utilize only its knowledge and speed.

It does not need your private data to generate answers for all possible scenarios we have shown you; you only need its speed.

Written by Ziyahan Albeniz (SOCRadar’s Technical Content Manager)