Tag: Error

Categories
Artificial Intelligence Journalism Software Engineering Tutorial

Evolving Errors – New Error Patterns In Remote Services, APIs, and Software With AI Agents

AI Agents Create New Types of Error Patterns in Remote Services, APIs, and Software

This article is about how AI agents manifest new errors that we have not previously been facing in computing and software engineering.

Why Are These Errors Novel?

Widespread use of authorized 3rd party AI agents interacting with your API or system remotely on behalf on consumers or businesses have not been widespread, high-volume, or using higher levels of threaded AI reasoning until the release and promotion of the OpenAI GPT Store.

With the introduction of conversational, threaded AI agents available to all consumers comes brand new ways errors can occur on your servers. Once the AI agents are calling to your services expect to see things that traditional functional algorithmic programming simply wouldn’t do. Some of these errors are similar to errors or attack vectors already in existence; the fundamentals of 1’s and 0’s still remain, but the novelty in this situation (novelty like new, not novelty like fun) is that these errors have reshaped and will manifest in strange new ways.

Let’s look at some generic ways AI agents manifest new types of errors in your server logs.

Error #1 – The Fake ID

Summary

When an AI exceeds its context window it may have dropped the tokens of system IDs it got from your server. On subsequent requests the AI will spontaneously generate incorrect type-correct remote system IDs.

Error Process

  1. User triggers AI agent to fetch the data of their latest post from the server.
  2. The server responds with the latest post and the latest post’s ID number.
  3. User uses AI agent to edit the content of the post.
  4. The action of editing the post causes the AI agent to exceed the input token context window and the post ID is dropped as a token.
  5. The user completes editing their post and instructs the AI agent to upload the edits to the server.
  6. The edited post content is sent to the server and in order to correctly form the request the AI agent generates a type-correct value for the ID.
  7. The request is rejected because the ID is incorrect.
  8. The AI agent is unable to adjust and fix the error as it no longer has access to the remote system ID, and it ultimately ends up in a failure state for the action.

Novelty

Previously computer programs did not spontaneously generate type-correct remote system IDs client-side.

Real-World Impact

    • You may accidentally overwrite entirely different objects or entities.
    • You may see a large increase in “incorrect ID” related errors in your server logs.
    • You may need to introduce AI directives to cache or store important IDs.
    • You may need to additional validation code.
    • You may need to additional confirmation flows.
    • You may need to introduce AI directives related to caching or repetition of of important IDs within a process to keep them in the token context window.

Error #2 – Acts of Creation

Summary

When an AI exceeds its context window it may have dropped the tokens of system IDs it got from your server.
On subsequent requests the AI will recognize that it does not have an ID and attempt to make a call to create a new entity or object. This can cause a number of issues depending on the type of object being created.

Error Process

      1. User triggers AI agent to fetch the data of their latest post from the server.
      2. The server responds with the latest post and the latest post’s ID number.
      3. User uses AI agent to edit the content of the post.
      4. The action of editing the post causes the AI agent to exceed the input token context window and the post ID is dropped as a token.
      5. The user completes editing their post and instructs the AI agent to upload the edits to the server.
      6. The AI agent recognizes it does not have an ID and calls to a creation endpoint, thereby creating a new article.
      7. The request is accepted and the post is duplicated.
      8. The AI agent is unable to get the original system ID and continues to spam the creation endpoint.

Novelty

Previously computer programs in editing mode did not spontaneously switch to a creation mode.

Real-World Impact

      • You may duplicate or recreate objects and data erroneously.
      • You may see a large increase in calls to creation methods.
      • You may need to introduce AI directives to cache or store important IDs.
      • You may need to additional validation code.
      • You may need to additional confirmation flows.
      • You may need to introduce AI directives related to caching or repetition of of important IDs within a process to keep them in the token context window.

Error #3 – Gobbledygook

Summary

An AI may correct generate part of a data structure for a request but may send along type-correct nonsense for the rest of the request.

Error Process

        1. User triggers AI agent to fetch the data of their latest post from the server.
        2. The server responds with the latest post and the latest post’s ID number.
        3. User uses AI agent to edit the content of the post.
        4. The action of editing the post causes the AI agent to exceed the input token context window and the post title is dropped as a token.
        5. The user completes editing their post and instructs the AI agent to upload the edits to the server.
        6. The AI agent recognizes it does not have a title for the article and generates a title to properly form the request.
        7. The request is accepted and the title is erroneously changed.

Novelty

Previously computer programs did not spontaneously generate type-correct nonsense client-side as part of request packet creation.

Real-World Impact

        • You may edit or overwrite data accidentally but not fail any type checks.
        • You may need to introduce AI directives to cache or store important parts of objects and entities.
        • You may need to additional validation code.
        • You may need to additional confirmation flows.
        • You may need to introduce AI directives related to caching or repetition of of important data within a process to keep them in the token context window.

Error #4 – Make Fetch Happen (aka AI Stampede)

Summary

If a request fails the AI will not back off from making that request again (unless you give it an explicit directive to stop retries.

Error Process

          1. User triggers AI agent to fetch the data of their latest post from the server.
          2. The server responds with the latest post and the latest post’s ID number.
          3. User uses AI agent to edit the content of the post.
          4. The action of editing the post causes the AI agent to exceed the input token context window and the post ID is dropped as a token.
          5. The user completes editing their post and instructs the AI agent to upload the edits to the server.
          6. The AI agent generates a post ID and it is incorrect.
          7. The request fails but the AI agent has no directive to stop retrying so it continues to make the request infinitely.

Novelty

Previously computer programs would fail on incorrect requests and not continue to retry them unless explicitly programmed to do retry requests.

Real-World Impact

          • You may DDoS your own server.
          • You may tie up your AI agents and models processing the same request over and over again.
          • You may need to introduce AI directives to cache or store important IDs.
          • You may need to additional confirmation flows.
          • You may need to introduce AI directives related to caching or repetition of of important data within a process to keep them in the token context window.
          • You may need to introduce AI directives to stop retrying requests.
          • You may need to introduce ways to terminate AI agents or request processes.

This Is Not A Comprehensive List of Errors

These are real-world errors I have encountered in my own work with AI agents including developing with ChatGPT Plugins, Custom GPTs, BabyAGI, and AutoGPT.

There are likely many more errors that can occur and these errors will manifest in their own novel ways depending on your systems.

Categories
Software Engineering Tutorial

Ubuntu + NodeJS + Puppeteer + Chromium (+ PHP 8.2): The Fixes You Need

Ubuntu + NodeJS + Puppeteer + Chromium Is Hard to Work With

I’m gonna make this as quick as possible because this is likely your 10th article on this.

The particulars of my setup, including using PHP to trigger a NodeJS command as the www-data user from apache2, probably don’t apply to you. That is OK. Most of these fixes having nothing to do with PHP but only occurred because of how PHP runs as a very limited user.

Situation

I wanted NodeJS puppeteer running a headless chromium-browser on an Ubuntu server on AWS EC2 (or any cloud) running a normal Ubuntu AMI but I was triggering the NodeJS script that executed puppeteer via PHP calling to the system so everything was running as the restricted apache2 user “www-data”.

That’s a lot of tech stack.

It Worked Before

My previous server image and server template were Ubuntu 20.x. Once upgrading to Ubuntu 22.x everything broke. It turns out its not just Ubuntu 22 but a lot of potential issues.

Fixes & Tips

Forget my situation we’re here to fix your problem.

Node Verison – Puppeteer Need At Least Node 18

You need at least node 18 to run puppeteer. You can check your current node version via `node -v`

Are You Using nvm?

You may be using nvm to manage your node versions, like many many people.

This can make your situation confusing. Because nvm is per user.

If your user trying to run puppeteer + chromium-browser has nvm you need to make sure nvm is using at least Node 18.

When nvm installs a new version of NodeJS for a user it doesn’t set that version to the default version. You have to do that manually. Here is a command which will do that & insure that every time that user tries to execute a NodeJS script it uses the version of NodeJS you want.

nvm alias default 18

Restricted Users, like www-data, Cannot Have nvm

Restricted users cannot have nvm. So how do they manage NodeJS versions?

Restricted users like www-data use the global node version. This is the version of node installed on the server for all users, similar to other packages you install via apt-get install.

You need to make sure this version of node is fully upgraded to Node 18 or greater as well.

npm install

You probably ran npm install in your project directory. It installed puppeteer and puppeteer installed a version of chromium-browser.

Things have now officially become a mess. Welcome to package hell.

Errors

Can’t Find Chromium

Error: Could not find Chrome (ver. 119.0.6045.105). This can occur if either 1. you did not perform an installation before running the script (e.g. `npm install`) or 2. your cache path is incorrectly configured (which is: /var/www/.cache/puppeteer). For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.

This was happening to me.

I tried to solve the issue by hardcoding my “executablePath” in the puppeteer.launch call. That made my situation worse: everything would hang and timeout without errors when I manually supplied the correct executablePath. Do not hardcode “executablePath”: ‘/usr/bin/chromium-browser’ in your puppeteer.launch config.

Fix:
You want to be running the set of packages you installed from your package.json and not any other versions. Duh, right? Well, that means you need to make sure you know where they are.

When you execute this NodeJS script from some other process you probably forgot to change directories as part of your system call. Even if you specified the full path to the script that actual process, running as the restricted user you may not actually be in the right place. If that’s the case… it will try to use the global install of puppeteer. If that exists it will try to find Chromium in some weird place and it never will. Check where you are and where NodeJS thinks it’s getting the packages from: 

pwd; npm root;

Whoops! You’re not in the right place! But look what we just did: we combined shell commands with that semicolon. Do the exact same thing in your system calls before invoking your NodeJS script as your restricted user then it will look in the right place.

cd /path/to/project; node index.js

That means you also probably want to remove any *global* versions of puppeteer you think you installed. Why? Because it’s very likely your restricted user may run the global version. The global package will run and suddenly your project will return an error on puppeteer.launch that it can’t find Chromium because it’s looking in some weird place. Perhaps you had installed it previously, who knows, ditch the global version if you can.

npm -g uninstall puppeteer

Remember, the /var/www/ is PHP specific, that may not apply to you.

Ubuntu Snap Issue

user.slice/user-1000.slice/session-270.scope is not a snap cgroup

system.slice/apache2.service is not a snap cgroup

This is happening to thousands of people. Why? Ubuntu 22. It shipped with a new type of package management called Snap which lets packages download as the complete packages rather than piling up dependencies. It also locks them off a little bit more. You don’t need to know or care about it right now because NO MATTER WHAT YOU DO YOU CANNOT FIX SNAP.

Why? Because this article is about me running on AWS EC2 or any other cloud hosting. You don’t have kernel access like this on the Ubuntu AMI! You can’t fix this even with a boot script. Don’t bother with DBUS_SESSION_BUS_ADDRESS and don’t bother with systemd.unified_cgroup_hierarchy=0 and don’t bother with any of it that’s for the schlubs running Ubuntu on desktop or neckbeards with kernel access.

Fix:
I’m very sorry but your only option will be to download the Debian version of chromium-browser from some random person’s package repository. This person is allegedly an engineer at Cisco and it’s all public and on the up-and-up but there’s any other way around it. By installing the deb version of chromium-browser it will let you use headless chromium-browser without snap or snap cgroups.

sudo apt remove chromium-browser
sudo snap remove chromium
sudo add-apt-repository ppa:saiarcot895/chromium-beta
sudo apt update
sudo apt install chromium-browser

Other Fixes for Random Errors

Those errors above are so gnarly down the rabbit hole I had to write this entire blog post. The rest of the errors are a cake walk.

Permissions: Add args To puppeteer.launch

You forgot the args for puppeteer.launch

const browser = await puppeteer.launch({
                "headless": "true",
                args: ["--no-sandbox", "--disabled-setupid-sandbox"],
            });

More Permissions

You may need to explicitly set the cacheDirectory on puppeteer.launch. You need to figure this location out yourself and make sure any restricted users have permissions to get at it.

const browser = await puppeteer.launch({
                "headless": "true",
                "cacheDirectory": "/path/to/my/.cache/puppeteer",
                args: ["--no-sandbox", "--disabled-setupid-sandbox"],
            });

I Just Saved Your Project.

You owe me a follow on Twitter: @kickiniteasy