In the early morning of March 15th (GMT+8), OpenAI officially released the latest version of their large language model system, GPT-4. OpenAI claimed that the birth of GPT-4 represents their latest milestone in scaling up deep learning. What surprises will GPT-4 bring us?



How powerful is GPT-4?

According to the introduction of OpenAI, GPT-4 is a large multimodal model, meaning that it can process images and up to 25,000 words.

What can prove GPT-4’s amazing capabilities? Look at the demonstration below.

If you ask GPT-4 what happens when the glove falls off, it will answer that it will fall on the wood and the ball will fly up. It is surprising that GPT-4 has logicial thinking!

You can simply draw a sketch of a website which can then be identified by GPT-4.

If you upload a photo of the sketch of your website to GPT-4, it can generate the corresponding HTML for you.

From these examples, we can see that GPT-4 is more stable and more creative than GPT-3.5. It can understand and conduct more complicated instructions.

In addition, GPT-4 has improved in content accuracy and logical ability over the previous version. In the Uniform Bar Exam, GPT-4 scores over 90% of the others while GPT-3.5 scores were just over 10%. GPT-4 scores 700 on the SAT Math test which is 110 higher than what GPT-3.5 scores. In other standardized tests, GPT-4 performs much better than GPT-3.5.

In the demonstration, GPT-4 took almost one to two seconds to identify a hand-drawn website image and generate the corresponding code in real time to create a website that was almost identical to the hand-drawn version.

Apart from regular images, GPT-4 can process more complex graphic information, including tables, test questions screenshots, essay screenshots, cartoons, and so on. It can give abstracts and bullet points of given papers directly.

It is so strong that people fear being replaced by GPT-4 and losing their jobs.

What happens if GPT-4 audits a smart contract?

Beosin published a research article about ChatGPT last December to see if ChatGPT can detect security vulnerabilities in smart contracts. You can click here to read this article.

On March 15, Conor Grogan, director of Coinbase, claimed that he input a smart contract built on Ethereum to GPT-4. It found security issues at once and even showed how to exploit the contract.

Conor Grogan said the contract was hacked in 2018 and he tried Euler's smart contract, but it was too long for GPT-4 to process. Conor Grogan believes that AI will eventually help make smart contracts safer and easier to build.

Some claimed that ChatGPT can detect the vulnerabilities of Euler Finance, which caused a loss of about $200M. You can click here to read this security incident.

However, is it easy for AI to detect security issues of smart contracts?

Similar to earlier versions, GPT-4 still has some limitations.

OpenAI teams say that GPT-4 is not entirely reliable and that there may be inaccuracies in its reasoning, "GPT-4 generally lacks knowledge of events that have occurred after the vast majority of its data cuts off (September 2021), and does not learn from its experience... ...It can sometimes make simple reasoning errors which do not seem to comport with competence across so many domains, or be overly gullible in accepting obvious false statements from a user. And sometimes it can fail at hard problems the same way humans do, such as introducing security vulnerabilities into code it generates.”

Thus, OpenAI reminds users should be very careful when using GPT-4, preferably with manual review and context, or users should avoid using it in high-risk situations.

ChatGPT VS Beosin VaaS, which one is better at contract audits?

Formal verification experts with Beosin say: "CHatGPT can learn the complex patterns of smart contracts and classify contracts from different dimensions. It can help static testings, increase the types of vulnerabilities that can be identified, reduce false positives and missing rates, assist attribute-oriented testing and verification techniques to effectively link to domain attribute libraries, and enable fully automated testing and verification through automated contract detection and attribute insertion. However, ChatGPT has trouble identifying complicated logical issues which are often related to the business model and require security experts to detect, constantly summarize and build domain attribute repositories to the security of the contract."

We also found that not all problems can be solved by ChatGPT. For example, many vulnerabilities still require a rigorous audit by security experts or the use of Beosin Vaas, a formal verification tool to find security issues.

Beosin VaaS is a powerful One-click platform for the formal verification of smart contracts. The accuracy of detection is more than 97%. The vulnerabilities are precisely located and professional repair suggestions are given. More than 80 items of common security flaws and functional logic defects of smart contracts are automatically detected. It supports auditing smart contracts built on EVM-compatible blockchains and WASM, which helps developers improve the security of their smart contracts.

For example, on March 15, we alerted that Locked Deal contract of Poolz Finance was attacked. The attacker called CreateMassPools function of Locked Deal contract and triggered an integer overflow vulnerability in the argument. We tested that the vulnerability could be detected by Beosin VaaS but not by ChatGPT.

ChatGPT can not detect the logic problem of K invariance issue either.

Since the actual exchange transfer operation of DEX such as Uniswap is implemented in the swap () function of Pair. In order to prevent attackers from calling the swap () transfer directly on the Pair contract over the Router contract, you need to check the K value in the swap () function of a Pair contract, which means that the K value in a Pair needs to be invariant after a swap. If the codes which are responsible for K invariance check have security issues, then the attacker can exchange most of the tokens in a Pair for a very small number of tokens.

By studying the K invariance issue, we summarize the characteristics of the problem and extract the common attributes of the problem for the use of Vaas tools. After that, we extract the contract information of 140,000 addresses on ETH and BSC by analyzing the nodes' information. These contracts are all similar contracts for swapping. There may be the K invariance issue.

In addition to using the formal verification tool Vaas, Beosin formal verification experts also abstract the security issues into reusable security attribute invariants which are tested and verified automatically by Beosin security engine. It is proved that these reusable invariants of security attributes can effectively find new subtle vulnerabilities in smart contracts. These are the parts that AI like ChatGPT can't replace.

As an article, The False Promise of ChatGPT published on the website of the New York Times on March 8 says, "Today our supposedly revolutionary advancements in artificial intelligence are indeed cause for both concern and optimism. Optimism because intelligence is the means by which we solve problems. Concern because we fear that the most popular and fashionable strain of A.I. — machine learning — will degrade our science and debase our ethics by incorporating into our technology a fundamentally flawed conception of language and knowledge."

However, without the "Chat" tag, GPT-4 will create more productivity. Please feel free to share your thoughts with us on AI tools like ChatGPT.

Beosin is a leading global blockchain security company co-founded by several professors from world-renowned universities and there are 40+ PhDs in the team. It has offices in Singapore, Korea, Japan, and other 10+ countries. With the mission of "Securing Blockchain Ecosystem", Beosin provides "All-in-one" blockchain security solution covering Smart Contract Audit, Risk Monitoring & Alert, KYT/AML, and Crypto Tracing. Beosin has already audited more than 3000 smart contracts and protected more than $500 billion funds of our clients. You are welcome to contact us by visiting the links below.

Contact

If you have need any blockchain security services, please contact us:

Website Official Twitter Alert Telegram LinkedIn