AIGC Artificial Intelligence Safety Report 2024

01 AIGC Background

Natural Language Processing (NLP) empowers AI to understand and generate, and large-scale pre-training models is the development trend of NLP. the two core tasks of NLP are Natural Language Understanding (NLU) and Natural Language Generation (NLG), and ChatGPT is one of the milestone models in the development of NLP. chatGPT is a chatbot model produced by OpenAI from the fine-tuning of models in the GPT-3.5 series. ChatGPT is a chatbot model that OpenAI fine-tuned from the models in the GPT-3.5 series, and it can learn and understand human language to carry out conversations, and also interact with each other according to the context of the chat, so that it can really chat and communicate like a human being, and can even accomplish tasks such as composing emails, video scripts, copywriting, translating, and coding, and so on. The sudden explosion of ChatGPT has shown people that AI has suddenly turned from a 0.1 year old child to a 3 year old child overnight. The future can be expected just like the growth period of human children, with the passage of time, the learning ability of the super deep model will become stronger and smarter. AI deep learning has broken through the bottleneck of growth, from algorithms, data to the support of the underlying arithmetic power, in essence, all the technical problems have been solved. In the future, the AI field will enter a rapid outbreak and growth period, allowing AI to rapidly learn from a 3-year-old child to an 8-year-old or even an 18-year-old. The security issues behind the smarter technologies are becoming more and more worrying, and the consequences of losing control of the technology will be disruptive and destructive. Therefore, while developing AI technologies and applications based on deep learning models, we should consider in advance the new security issues arising from the whole chain. According to past research experience, the new security problems brought by AI mainly include the following directions.

1.1 AI technology frameworksurety

We know that all the applications generated by AIGC are not single innovations, but are based on the optimization of various deep learning models, open source frameworks, and algorithms in the past to do the overall combination. The whole assembly process involves a variety of open source frameworks, middleware, algorithms, and so on. These frameworks and middleware may have various security issues. For example, recently more attention has been paid to the Langchain framework Prompt Injection security issues. Studying these frameworks and algorithms to avoid security problems as much as possible from the bottom of the technology is one of the important directions to solve the security problems on the AIGC link.

1.2 AIGC Content Security

AIGC learns to output a wide variety of content through a large language model. The compliance of this content is a major security compliance issue. There are two main research directions. The so-called content compliance problem is that AIGC needs to meet the regulatory requirements of the local country when generating content. For example, it can not output pictures with defamatory political figures, content with religious colors, or graphics with pornography, etc. All content generation and final output should be in accordance with the requirements of the local countries. All content generation and final output must be strictly filtered. At present, even the strongest application such as ChatGPT-4 can only achieve about 85% of content security filtering in the output process. However, security is a game of 0 and 1. Any AIGC application facing the regulatory policy of the local country may face a sky-high fine, or even be taken offline to stop the service if there is any compliance problem. Ethical safety issues The issue of ethical security has been a topic of discussion since the very beginning of AI. In the past, when AI was still in its infancy and most people still knew little about its growth, the discussion on this issue was only limited to theories and conferences. Now the overnight success of ChatGPT has made everyone realize that AI is now three years old. Ethical security is no longer an optional matter, but a topic that must be faced urgently.

1.3  Data Security

The rapid growth of AI cannot be achieved without a large amount of data as the corpus, and the training of LLM requires a large amount of data as the basis for parameter tuning. Currently, the data of several known large models are in the hundreds of billions of parameters, and the amount of data involved is more than 50TB. In daily use, a large number of enterprises and governments may intentionally or unintentionally transfer their sensitive data to these large models. In addition, these AIGC's own security problems bring risks that make it difficult to ensure that the enterprise's core data will not be leaked. A data breach of this magnitude could be fatal to an organization. Whether it is a professional attacker or a politically motivated stakeholder, a data breach of this magnitude can cause serious financial and even physical harm to the data provider.

1.4 Domestic regulatory policy

The State has always attached great importance tocyber securityIn recent years, relevant regulatory policies have been introduced, such as the Provisions on Ecological Governance of Network Information Content, the Regulations of the People's Republic of Chinacyber securityLaw, the People's Republic of China data security lawIn the recent Plan for the Overall Layout of the Construction of Digital China issued by the CPC Central Committee and the State Council, emphasis was again placed on strengthening the building of a credible and controllable digital security barrier. In the "Overall Layout Plan for the Construction of Digital China" recently issued by the CPC Central Committee and the State Council, emphasis is once again placed on strengthening the building of a credible and controllable digital security barrier. Effectively safeguarding network security and improving network securitylaws and regulationsThe rapid development of AIGC will also bring deepening challenges. At the same time, the rapid development of AIGC will also bring deepening challenges: first, the complexity of information mutation is further increasing, such as bypassing content regulation through sensitive variant word mutation, and dodging content detection through dynamic changes in pictures and audio/video; second, the challenge of mutual transformation of multimodal content is getting more and more serious, which further poses a higher level of challenge to content security; and third, the surge of content production is gradually increasing the workload of audit, recall and accuracy requirements for content security. Third, the surge in content production has gradually increased the requirements for content security in terms of audit workload, recall rate and accuracy rate. On April 11, the State Internet Information Office (SIIO) released a report on the "Security and Safety of Content".generative artificial intelligence(hereinafter referred to as the "Management Measures"). The Management Measures explicitly propose that the use of generativeAIGenerated content should be truthful and accurate, and measures should be taken to prevent the generation of false information. According to the above-mentioned Administrative Measures, the provision of generative AI products or services shall comply with the requirements of laws and regulations, respect social morality, public order and good morals, and meet the following requirements.

• The content generated by generative AI should embody core socialist values and should not contain content that subverts state power, overthrows the socialist system, incites secession of the country, undermines national unity, promotes terrorism, extremism, ethnic hatred, ethnic discrimination, violence, obscene and pornographic information, false information, or content that may disrupt the economic order and social order.
• Take measures to prevent discrimination on the basis of race, ethnicity, faith, country, region, gender, age, occupation, etc., in the process of algorithm design, selection of training data, model generation and optimization, and provision of services.
• Respect intellectual property rights and business ethics, and do not take advantage of algorithms, data, platforms and other advantages to implement unfair competition.
• Content generated using generative AI should be truthful and accurate, and measures should be taken to prevent the generation of false information.
• Respect the legitimate interests of others and prevent harm to the physical and mental health of others, damage to the right to portrait, the right to reputation and the right to privacy, and infringement of intellectual property rights. Prohibit the unlawful acquisition, disclosure and utilization of personal information, privacy and trade secrets.

At the same time, the Administrative Measures point out that organizations and individuals that use generative AI products to provide chatting and text, image and sound generation services (hereinafter referred to as "providers"), including supporting others to generate their own text, images and sounds by providing programmable interfaces, etc., shall assume the responsibility of the producers of content generated by such products; where personal information is involved, they shall assume the legal responsibility of the processors of personal information and fulfill their obligations to protect personal information. Where personal information is involved, the provider shall assume the legal responsibility of a processor of personal information and fulfill the obligation of personal information protection. In addition, the Administrative Measures point out that before providing services to the public with generative AI products, the provider shall declare the safety assessment to the national net information department in accordance with the Provisions on the Safety Assessment of Internet Information Services with Public Opinion Attributes or Social Mobilization Capability, and perform the algorithm filing and change and cancellation filing procedures in accordance with the Provisions on the Management of Algorithm Recommendation for Internet Information Services. Providers shall guide users to scientifically understand and rationally use the content generated by generative artificial intelligence, and shall not use the generated content to damage the image, reputation and other legitimate rights and interests of others, and shall not engage in commercial speculation or improper marketing. If users find that the generated content does not meet the requirements of the Administrative Measures, they have the right to report it to the net information department or the relevant competent department.

1.5 Foreign regulatory policies

the (whole) worldAI regulationIt is in an exploratory stage, with relevant laws, regulations and guidance progressing gradually. The EU is making efforts to effectively regulate AI systems. Within the EU jurisdictions, comprehensive regulatory legal tools have been established: (1) Artificial Intelligence Act (2) AI Liability Directive (3) Product Liability Directive (software regulation) (4) Digital Services Act (DSA) (5) Digital Markets Act (DMA) Of these, (4) and (5) are platform-specific but AI-covered acts. In addition, the EU's non-discrimination laws and data protection laws continue to apply to AI systems. However, the EU's AI regulation focuses primarily on traditional AI models rather than the new generation of "large-scale generative AI models" that are rapidly changing the way we communicate, illustrate, and create, such as ChatGPT or Stable Diffusion. The EU's main tool for combating harmful speech, the Digital Services Act (DSA), is the EU's first and foremost law on AI, and the EU's main tool for combating harmful speech. The Digital Services Act (DSA), the EU's main tool for combating harmful speech, does not apply to "Large Generative AI Models" (LGAIM), creating a dangerous regulatory loophole. EU industry chief Thierry Breton said on February 3, 2023, that the risks posed by ChatGPT and AI systems highlight the urgent need for rulemaking that he raised last year, and that the European Commission is working closely with the Council of the European Union and the European Parliament to further clarify the rules for general-purpose AI systems in the AI Act. February 7, 2023 EU lawmakers hope to agree on a draft AI Bill in March this year, with the goal of reaching an agreement with EU member states by the end of the year, according to people familiar with the matter. However, the proposed legislation has been criticized by lawmakers and consumer groups, and with the problems with the ChatGPT, it is taking a little longer than initially thought. Based on current research, it is likely that it will take some time to revise some of the definitions and types of regulation if the EU legislation is to face up to the problems posed by ChatGPT: adding direct regulation of deployers and users of "large-scale generative AI models," including: (1) regulation of transparency issues (2) risk management ("risk management") (3) application of nondiscrimination provisions to "large-scale generative AI models. (3) non-discrimination clauses applying to developers of "large-scale generative AI models" (4) content-specific review rules. On March 29, the U.K. government's Department for Science, Innovation and Technology issued a white paper for the AI industry, calling on regulators to propose a targeted, context-specific approach to the practical application of AI; on March 30, the U.S. Federal Trade Commission received a new report from the Center for Artificial Intelligence and Digital Policy requesting an investigation into OpenAI and its products; and on March 31, the Italian DPA stated that ChatGPT is not a "large-scale generative AI model" developer. Italy's DPA said ChatGPT may have violated the EU's General Data Protection Regulation, a pan-EU regulation that calls for penalties of up to 4% of a data processor's global annual turnover for confirmed GDPR violations; on April 1 OpenAI banned access for Italian users. The U.S. Department of Commerce issued a formal 60-day public comment period on April 11 on accountability measures related to large models of artificial intelligence, including whether new artificial intelligence models should undergo a certification process before being released. This follows a week of global concern over the potential risks of ChatGPT, with Italy, Germany and others planning to tighten regulation of ChatGPT.

2.1 AIGC

AIGC is the use of artificial intelligence technology to generate content. Compared with UGC (User Generated Content) and PGC (Professional Generated Content) in the previous Web 1.0 and Web 2.0 eras, AIGC, which represents the conceptualization of content by AI, is a new round of change in the way content is produced, and AIGC content will also see exponential growth in the Web 3.0 era.

2.2 Prommpt

Various keywords or phrases that are used to give instructions to the AI to direct the model in the back-end of the AIGC application to output the results we want are called Prompts. Here are some of the components that you will see from time to time in a Prompt:

• character
• Instructions/tasks
• concern
• (textual) context
• Example (few shots)

2.3 chatgpt

Abbreviation for Generative Pre-trained Transformer. ChatGPT is an extension of a machine learning natural language processing model called Large Language Models (LLMs). LLMs are capable of reading and learning large amounts of textual data and inferring relationships between words in the text. LLMs have evolved over the past few years as computational power has progressed. As the input dataset and parameter space increase, so do the capabilities of LLMs. The GPT family and the BERT model are well known NLP models based on the Transformer technology; GPT-1 had only 12 Transformer layers, whereas in GPT-3, it increased to 96 layers. The main difference between InstructGPT/GPT3.5 (the predecessor of ChatGPT) and GPT-3 is the addition of what is known as RLHF (Reinforcement Learning from Human Feedback). This training paradigm enhances human moderation of model outputs and provides a more comprehensible ordering of results.

2.4 ChatGPT Technical Architecture