Vietnamese AI experts say there is something more worrying than the prospect of AI destroying humanity

(Dan Tri) - According to expert Nguyen Chien Thang, the most worrying AI threat is not the prospect of robot rebellion, but when humans give technology power beyond control.

A robot that was pushed down suddenly stood up and attacked a human. Another machine shows fear, trying to resist when being put into the furnace for destruction. Such videos appear on social networks, attracting attention because of their realistic images and scenarios that are no different from science fiction movies.

What they have in common is that they evoke a long-standing fear: What will happen if artificial intelligence (AI) becomes so intelligent that it no longer accepts human control?

That worry is even more noticeable when leaders of the world's leading AI companies such as OpenAI and Anthropic mention the serious risks that this technology can pose to humanity.

In an interview with Dan Tri reporter, Mr. Nguyen Chien Thang, Director of Development Center, Saigon - Hanoi Commercial Joint Stock Bank (SHB), shared his perspective on the real risks of AI and how people control this technology.

Recently, videos of robots protesting when being destroyed or attacking people after being pushed down have appeared on social networks, making many people fear the prospect of an AI rebellion. In addition, leaders of OpenAI and Anthropic also issued warnings about the risk of AI causing disasters. How do you assess these concerns?

- First of all, the robot videos we just mentioned are actually products created using AI. Current technology can create very realistic images, making viewers think that robots have emotions, and even have the ability to resist humans.

Regarding warnings that AI can cause disaster for humanity, in my personal opinion, some messages are being conveyed in a direction that is too far from reality, partly to attract public attention to technology businesses.

However, if asked whether AI is dangerous or not, my answer is yes.

What is of concern is not that AI naturally forms consciousness to fight humans, but rather the risks when the system is given inappropriate action capabilities and access rights.

Can you explain more specifically how AI can be dangerous, especially in the context of Agentic AI growing strongly?

- Previously, AI mainly performed tasks such as writing text, summarizing information or answering questions. However, with the emergence of Agentic AI (AI capable of acting autonomously), technology has taken another important step forward.

Agentic AI can plan, select tools, perform multiple work steps and analyze results to complete assigned goals.

For example, an AI system equipped with a security vulnerability search tool could try to connect to an IT system, analyze the response, detect weaknesses, and then continue using another tool to try to gain deeper access.

These capabilities can endanger information systems. However, it should be noted that AI must use tools to perform those actions.

So who provides the tools for AI? It's people.

An AI model, no matter how smart, cannot naturally access banking systems or corporate data if it is not connected to an application programming interface (API), computer network or appropriate execution tools.

It is humans who grant permissions and create conditions for AI Agents to perform actions on the system.

So, instead of just asking how dangerous AI can become, we need to be more concerned about who is controlling the actions that AI is allowed to perform.

Currently, many people use AI assistants on their computers and accept permission to access files, folders, and even all personal data so that AI can work more conveniently. What potential risks does this pose?

- The tools given to AI are like a double-edged sword.

If you do not give AI permission to read or edit files, it will be difficult for the system to perform the tasks the user wants. But if permissions are not granted appropriately, the AI ​​can perform unintended operations.

In fact, there have been cases where users gave too broad access rights, causing AI to mistakenly delete a series of files during work processing.

The cause does not necessarily come from sabotage intentions. Sometimes, AI misunderstands the request or chooses an inappropriate processing option within the scope of its granted authority.

So what do users need to do to both take advantage of AI's automation capabilities and ensure data security?

- In information security, there is a very important principle called Least Privilege (minimum authority). AI should only be given access and use to tools that are absolutely necessary for the task.

This principle is similar to how businesses delegate authority to employees. A clerical employee does not need access to the vault or accounting system if the job is not relevant.

The same goes for AI. For example, when I use an AI assistant to process documents, I only grant access to a specific folder, instead of all data on my computer.

For sensitive operations such as deleting a series of files or performing transactions, the system must have a human-in-the-loop confirmation step before execution.

In AI development, there is also the concept of Harness, which can be imagined as a set of reins used to control a horse.

For AI, Harness is a system of tools, rules and control mechanisms that help limit the model's behavior. Without the necessary barriers, AI can malfunction, affecting connected systems.

It's important to leverage the power of AI but establish the right scope of operations, rather than delegating unnecessary authority.

If you divide AI risks into two groups, including existing risks and risks that may occur in the future, which do you think is more worrying for Vietnamese users?

- In my opinion, the immediate concern is that many people today do not have enough knowledge about AI to use technology effectively and safely.

An example is the situation where employees accidentally submit financial reports, customer information, intellectual property or business secrets to public AI chatbots for analysis, leading to the risk of information leakage.

Another risk is inappropriate decentralization, causing AI to mistakenly delete data or affect system operations.

In addition, many people lack the skills to recognize Deepfake images and videos or authenticate fake calls, thereby easily becoming victims of fraud.

These are all manifestations of not having enough AI Literacy - the capacity to understand, use and evaluate AI safely and responsibly.

Regarding the future, AI technology is changing at such a rapid pace that it is difficult to make firm predictions.

However, as Agentic AI becomes increasingly capable of acting autonomously, large language models (LLMs) can still produce inaccurate results or act inconsistently with human goals.

Therefore, improving AI knowledge and building appropriate control mechanisms needs to be done now.

In a banking environment, where customer data and transaction information have very high security requirements, how is AI managed and decentralized to limit risks?

- First of all, we still apply the Least Privilege principle, which means only granting necessary permissions for each task.

The important point is to manage AI as an official electronic employee (e-employee), with clear identity and authority limits.

Each AI agent is given a unique identifier, similar to an employee's account when accessing the banking system.

At the same time, each AI Agent has an Agent Card, recording information such as time of creation, function, management unit and scope of authority.

When AI accesses data or performs operations on the system, access rights must also be checked similarly to an employee.

For example, administrative staff do not have permission to access the vault or make money transfer orders on the Core Banking system. An AI Agent not assigned those tasks is also subject to corresponding limitations.

Besides decentralization, how does a bank choose an AI model to ensure that sensitive data is not leaked out of the system?

- Another important principle is to use appropriate large language models (LLMs) based on classifying data according to sensitivity level.

For general analysis tasks that do not contain confidential information, we can take advantage of AI models on the public cloud computing platform (Public Cloud) to exploit their processing and thinking capabilities.

On the contrary, sensitive data related to customers, accounts and banking operations need to be processed on internal AI systems (Local AI/On-premise).

For specialized business problems, we can use small language models (Small Language Models - SLM) customized to perform a specific task, such as classifying text or checking the validity of records.

This approach helps leverage the capabilities of different AI models, while ensuring data security and management requirements.

With AI systems that can perform many work steps on their own, where is the limit that banks should not let AI make the final decision?

- In my opinion, the division should not be based on the field but should be based on the risk level of each business problem.

For low-risk tasks such as summarizing meeting minutes, synthesizing information or making reference recommendations, AI can do it automatically.

At a medium risk level, such as composing a customer care email, AI can prepare the content but needs to wait for employee approval before sending.

Particularly for high-risk operations such as credit appraisal and loan approval, AI should only play a supporting role in synthesizing, collating and analyzing information.

Final approval and disbursement decisions must still be reviewed and responsible by competent officials. This is a case where human participation is needed in the decision-making process (Human-in-the-loop).

When an AI system makes a wrong decision, affecting customers or causing damage to the business, who will be responsible: the user, the developer or the organization deploying the AI?

- This is an important issue in AI Governance.

When implementing AI into practice, businesses must clearly define the responsibilities and authorities of each department from the beginning.

AI Agent is a technology system and cannot be held responsible before the law. Responsibilities must be assigned according to each operational stage.

If the error comes from programming logic, the responsibility of the technology development unit should be considered. If the input data is incorrect, the data administration and supply department must be checked.

If an error arises during the approval or performance process, responsibility belongs to the competent individual or department according to the applicable process.

In particular, before putting AI into operation, relevant departments must participate in testing, acceptance and confirm responsibility for their scope of work.

Specific legal liability also depends on the law, contract and nature of each incident.

However, AI is sometimes compared to a "black box" because the processing process is very complex. If it is not possible to clearly explain why the AI ​​made the wrong decision, how can responsibility be determined?

- In AI Governance, there is a very important concept called Observability (the ability to observe and monitor activities).

When AI does work, the system must be able to log the steps that have taken place, from the moment the request is received, the data source retrieved, the tool used until the final action.

For example, if AI sends the wrong email to a customer, engineering can retrieve logs by AI agent, time period, and related transaction.

From there, we can determine where the AI ​​got the information from, what tools it used, and what actions it took to localize the cause.

This is an important basis for monitoring, troubleshooting and determining responsibility during AI operations.

Thank you for taking the time to share!

Content: Doan Thuy

Design: Tuan Nghia