Coinbase: ChatGPT Doesn’t Meet Platform’s Security Review Process

Coinbase used ChatGPT to perform automated token reviews utilizing the platform’s ERC20 token review framework and compared the results with standard token security assessments.

The Blockchain Security team leverages in-house automation tools developed to assist security engineers in reviewing ERC20/721 smart contracts at scale.

Coinbase compared 20 smart contract risk scores between ChatGPT and a human security assessment, and ChatGPT produced the same results as a human assessment 12 times. But ChatGPT mistakenly marked high-risk assets as low-risk in five out of eight misses.

While ChatGPT is different from the exact standard to clearly demonstrate that it should be integrated into Coinbase’s content review process, it has shown enough potential for the platform to continue with further research.

The platform says once automated or intelligent processes can demonstrate the same level of accuracy as their existing processes, these tools will only be used as an aid, not an aid, not the main tool.

Conor Grogan, director of cryptocurrency trading platform Coinbase, has included an Ethereum contract directly into the latest version of the popular chatbot, GPT-4, and it has highlighted numerous security holes and areas surface area where smart contracts can be exploited, he said in a Twitter post on March 14.

I dumped a live Ethereum contract into GPT-4.

In an instant, it highlighted a number of security vulnerabilities and pointed out surface areas where the contract could be exploited. It then verified a specific way I could exploit the contract pic.twitter.com/its5puakUW

— Conor (@jconorgrogan) March 14, 2023

Grogan posted a screenshot of the AI bot’s analysis, which seems to show that ChatGPT can pinpoint critical issues and vulnerabilities, as it concludes that the smart contract analyzed should not be used, as it contains serious vulnerabilities and is based on an illegal scheme.

Previous smart contract tests on the first version of ChatGPT were also able to detect code errors, but it only stopped to a certain extent.

DISCLAIMER: The Information on this website is provided as general market commentary and does not constitute investment advice. We encourage you to do your own research before investing.

Source