Categories
Growth Marketing Software Engineering Twitter

Twitter API Now Costs Money… The Price is an Expensive Failure of Twitter Ads

The New Twitter API Pricing Structure is an Expensive Failure of Their Ads Product

The new Twitter API pricing structure that was announced earlier today is incredibly expensive and will kill off most smaller apps. According to this tweet by the official TwitterDev account on February 9th, 2023 Twitter will begin charging developers to use their API for applications. The pricing for the Twitter API (as far as anyone knows at this moment) is located at this link.

TRIGGER WARNING:If you are a developer who has built an app on the Twitter API this pricing structure is going to give you minor heart palpitations.

Which of the Twitter APIs Will Cost Money?

Both versions of the Twitter API, including Twitter API 1.1 and Twitter API 2.0, will cost money to use.

Will This Stop Developers From Using the Twitter API?

Absolutely. Most Twitter apps use the free usage tier because realistically Twitter apps are not profitable. They are small scale social apps that are used for growing your main application’s user base or are just for fun. Even more serious Twitter apps or apps that use major data access for larger data mining are hardly profitable and rely on the extremely low cost or free access to the data to continue to service Twitter users on the platform.

Is There a Free Tier For Small Apps?

Yes… but it is only for Sandbox apps (meaning the applications are not live for most users) and the current request limit is 250 requests PER MONTH. The average development cycle of a small app even for an experienced developer will make hundreds of small API calls during the integration process to debug their integration and insure that it is working correctly for users. This includes not just the creation of the app but automated testing of the integration to make sure it stays working as other code around it changes. This will make it incredibly difficult to even create new Twitter apps as a developer.

Argument: The Servers Cost Money. Twitter is Losing Money. Twitter Needs to Charge Money

This is an incorrect argument based on a simplification of how a hyperscale social network like Twitter works as a business. Yes, the Twitter Platform is a major cost center for Twitter. However, the Twitter API (both 1.1 and 2.0) currently have many, many limitations on usage for apps that we previously much more permissive and allowed much larger tiers of free usage. Could Twitter have started charging between $10/mo and $100/mo based on usage for the API and it would have been OK? Most likely. But these new prices are so far out of line with the economic models of Twitter apps and of course are unlike the rest of their industry.

If you look at other competing platforms like Facebook, Instagram, or Google they all offer their APIs with some restrictions but they give extremely large amounts of usage and of course all their APIs are free to use. How do Twitter’s competitors keep offering their APIs for free with such large amounts of usage? Ads.

Twitter Charging Extremely High Prices for their API is a Failure of their Ads Product

It’s no secret the ads on Twitter are terrible. This is well known to anyone who has used Twitter extensively for many years as a normal person. On the other side of things, the advertisers who are making Twitter ads and paying for user clicks, have known for many years that most Twitter ads are not profitable and do not “back out” (this is a marketing industry term meaning the ads are a profitable way to spend money). Buying Twitter ads is rarely a high priority in any marketing department and is usually an afterthought. How does a failure of the Ads product result in Twitter needing to charge so much for their API?

Without an excellent ad product Twitter is not able to generate enough revenue to support the business like their competitors do. This results in the API being a highly successful product and growth strategy while being a major cost center that Twitter cannot pay for.

What is Elon Doing About Ads?

After Elon’s takeover of Twitter he had a major focus on the ad business because Twitter is bleeding cash horrendously. Unfortunately he lost a lot of advertisers due to the type of contentious content he hyped on the platform. He spent a lot of time in the Spaces he was in talking about the display ads and how to make them “relevant” to users hoping that there were secrets to magically making ads more relevant to users with technology.

Can Twitter Make Better Ads?

It may be too late for this. Elon lost many of the larger ad purchasers when he started treating Twitter as his personal playground rather than acting like a CEO running a failing business. This was a major blow to existing ad revenue stream. On top of that he was searching for answers to the “relevance” problem looking at things like machine learning or AI; unfortunately those avenues alone will not provide the substantial additional revenue. His primary focus was to show “products that people want to buy”; Elon was clearly hunting for the Instagram ad revenue model. He should be more focused on the Facebook style ad methodology that doesn’t have quite the focus on pushing to ecommerce

Elon also had the ads team try to surface more ads on the app. That may have increased revenue slightly but without excellent ad inventory to display that could actually make users MORE ad blind.

Are There Simple Solutions to the Twitter Ad Problem?

The easy solutions to improving ads and their relevance have so far been ignored. The only two simple solutions that Twitter could take overnight would be:

1. Push the text that says “Promoted” to the top of the ad unit (exactly like Facebook, Instagram, and Google do.) This seems counterintuitive at first: wouldn’t that REDUCE clicks on ads since users know they are ads? Yes and no. The problem with users not knowing something is an ad is that it incentivizes and optimizes for clickbait ads. Advertisers want you clicking on their ad before you know it is an ad. This makes advertising and advertisers on Twitter heavily focused on clickbait style ads rather than creating high value adds and trying to find relevance to users with quality ads that drive clicks. This is a very, very minor fix engineering-wise but it would also require shifts in the overall ad ecosystem (ad content & advertisers shifting) before it would pay off. This may not be possible given Twitter’s short runway.

2. Include the profile pictures and usernames of any users that interact with the add below the ad but above the tweet metrics. This includes any followers, following, 2nd degree followers or following, celebrities, or major brands that have liked, replied, or view the original tweet or who follow or engage with primary account that created the ad. This gives social pressure to the ads which would greatly increase their relevance to most users.

Kick You Are Such a Whiner, Give Us Big Solutions

Again, unfortunately it may be too late to save Twitter’s revenue via ads. There were two major plays Twitter needed to drive at using a combination of the the API + Platform + Ads team over the last years and they have failed to create the necessary products that would allow them to feature match with competitors and get the level of ad revenue they need.

Twitter Connect – The Missing Login Link

1. “Twitter Connect” There is no comprehensive web Javascript library similar to Facebook Connect for Twitter that works well. There is a JS lib that they supply which is mostly used for correctly rendering tweets but it isn’t pushed heavily as a login solution. If they had pushed more aggressively for using the JS lib as the standalone login library for applications this would have given them a better contextual understanding of what people are looking at *when they are NOT on Twitter* and that is a large part of Facebook’s strategy around understanding ad targeting. This would have also unlocked much more powerful ad campaigns particularly around retargeting which is a major focus of ecommerce advertising.

This would not be a huge technical effort for their Platform and API engineering team even after the Elon layoffs but unfortunately it relies on external website adoption. With the increase in the API costs and level of desperation to drive revenue it’s unlikely that even creating this JS lib would get the adoption to reach critical levels on external websites to heavily impact ad revenue.

“Twitter AdSense” – The Ad Platform That Never Was

2. “Twitter AdSense”. Twitter doesn’t get as much traffic as their competitors (Facebook, Instagram, Google Search, GMail) and so they can’t realistically supply the same level of ad inventory as their competitors. In order to drive more ads from their ads platform Twitter needs to be showing ads on external website to get the additional impressions. This is how the Google AdSense model works

You can see this model in action on this blog. I include a Javascript library in the header of this blog supplied by Google AdSense which automatically places ads in the spaces I tell it to. If you, dear reader, click the ads then the advertiser pays Google $1 and then Google pays me $0.20. If 1000 people view a blog post I also get about $0.02.

If Twitter had a similar Javascript library it would have been possible for some websites to adopt their ads on their website. Advertisers simply don’t have this option with Twitter. It is not a trivial thing for their Platform + API + Ads team to build and so with recent layoffs I do not think they would reach a level of product that would allow them to compete with Google AdSense.

Twitter is in a Bad Position, Hence the Twitter API Prices

As you can see from this blog post the new Twitter API prices are not Elon trying to get rid of bots. The new pricing does reflect Elon trying desperately to turn Twitter in to less of a money burning machine: it makes sense to charge for the API. But the actual level of the pricing, far too expensive or exorbitant for almost all of the apps to actually pay for, is actually a major failure of the overall situation of ads on Twitter and has nothing to do with the very successful Twitter API and Twitter Platform.

Don’t Blame the Twitter API or Twitter Platform Team

It isn’t their fault they made an incredibly successful product that may be the last viable source of new revenue for a desperate Twitter.

No AI or ChatGPT was used in the creation of the blog post. All spelling errors or grammatical mistakes are mine.

Categories
Artificial Intelligence DrawGPT Journalism Software Engineering

DrawGPT – Make AI Art & Draw Images Using An AI That Only Knows Text


Use DrawGPT to Draw Anything With an AI… Using Only Words

I recently created a new way to generate AI art that does not directly use or copy artists work to generate images and is an exploration in how to visually enable large language models (LLMs).

Click this link to try it out and see what you can draw and get a sense of what the app is like.

How Can an LLM Know About the Visual World?

I was interested in how ChatGPT was able to understand the visual world despite being an AI that is only trained on text and words. It does not use any images, how does it know what things look like?

How can an AI that has never seen an image, had no images in it’s training set, and cannot output an image know what the visual world looks like?

I spent a few days puzzling over this and came up with a solution that I think is pretty cool and offers a nice proof that LLMs can become visually enabled.

DrawGPT – An Exploration in Visually Enabled LLMs

After thinking about how to get an AI LLM to render images I decided instead of just a proof of concept I would try to create an entire application that would showcase exactly how this could be done.

You can see it here at this link DrawGPT.

How Can an LLM Become Visually Enabled to Generate Pictures and Images?

The first step in creating a visually enabled LLM is of course the training data.

In my experience with ChatGPT I found that it was highly likely OpenAI had in fact use CLIP or CLIP-like data in their training data for GPT-3. It would be very difficult for a large language model to have an understanding of visual objects, their color, relative visual compositions of an objects, and everything else based on purely textual information alone.

While I cannot prove definitively this is true it seems likely given OpenAI’s products like DALL-E.

There is certainly a lot of visual information in large language model training sets that use only text. Paintings like the Mona Lisa are discussed in depth in art reviews, basic anatomical structures of things like animals are discussed in biology textbooks, things like buildings and skylines and landscapes are written about endlessly in literature. But I do not believe that would be enough to enable an LLM to become visually enabled in a way that would consistently output correct visual imagery.

CLIP, (an AI program that can take an image as an input and create a text description of that image), is a tool that can take visual text descriptions to the next level. By breaking down a visual image in to distinct text tokens CLIP and CLIP-like data creates a direct set of tokens related to visual imagery.

We know CLIP data works very well for creating AI art and generating images with AI because things like Stable Diffusion and Midjourney and DALL-E all use CLIP or CLIP-like data to generate images. This hinted me towards a direction for DrawGPT.

Text Tokens, Pixel Data, and Diffusion, Oh My!

Most of the AI art tools we see right now (Jan 2023) are based on a combination of CLIP data to create text tokens and latent pixel diffusion. This is what allows “text to image” AI art.

In order to be able to create “any” image these pixel diffusers need to be trained on copious amounts of images which get their subject matter extracted either by metadata provided in the training set or by running images in the training set through CLIP and using the output alongside the image.

What is going on behind the scenes with text inputs to pixel diffusion is that the text tokens are actually parsed to create the sampling distribution for the pixel diffusion. It breaks down the text phrase you sent as an input and then starts sampling random pixels based on the text tokens and the more times it can go through and take guesses as to what pixel goes where the better the output image is.

This is a phenomenal way to create AI art and it is very effective. But it also has some major issues.

The major problem with things like DALL-E and Stable Diffusion is that the image sets they were trained on did not necessarily credit the artists properly. Things like the artists style, the subject matter, the image composition, and many more things were extracted during the training using CLIP or available metadata.

And we’re not talking about a few images here. We’re talking millions of images scraped from the Internet and possible from sources that did not even know they were being scraped. Yes technically the terms of service were not broken during the collection of the images for the training set but obviously the resulting backlash suggests that the image collection was in an ethical gray zone.

As we’ve seen online there are many artists who are not happy with the way their work is being used in these AI art tools.

This is a major issue and it is something that I thought I could also uniquely address with DrawGPT by using ONLY an LLM… no actual pixel data. An LLM cannot copy anything about an artists work directly because it is not sampling or reading the pixel data of the images, only the text descriptions of them from CLIP data.

DrawGPT – Part of the Solution to Potential Art Theft & Ethical Dubiousness

One way to easily get around the issue of artists not feeling that their work was being copied is simply to not copy it.

That seems simple enough on the surface but in practice has not really been realistic. With the introduction of genuinely large LLMs like GPT-3, GPT-3 DaVinci, ChatGPT, Bloom, and others the total corpus of textual works in the training set, including any CLIP data, should be proficient to give enough visual references for an LLM to be able to create images simply from words.

The problem is that the LLMs are not trained to create images. They are trained to create text. And while they can be trained to create images they are not trained to create images in a way that is visually coherent.

And that is where the question of how a visually enabled LLM is able to express itself. While it may know what a dog is, it may not know what a dog looks like. It may know what a dog is & it may know what a dog looks like from written examples how would it draw given that it cannot output pixel data?

How Can An AI LLM Draw?

This was my first question. Because the field of AI research with these LLMs, transformers, and diffusers is so new it wasn’t really something AI researchers were looking at. I did not have a lot of work to reference as no one had really been considering how to get the LLM itself to draw.

Much like the need for a truly massive training set the LLMs themselves needed to reach a certain maturity before it was realistic to explore for some research.

Even if the AI LLM has enough visual reference data it also requires an AI LLM with sufficiently large corpus of training data on an output medium to enable the ability to output tokens correctly enough that images could be rendered.

With the introduction of GPT-3 and the checkpoint GPT-3 DaVinci we have reached a point where the AI can in fact command a visual medium with enough complexity to correctly render images.

What is the medium for an LLM? Well, seeing how it can only use text it needs the text that it outputs to create an image. Since the images are digital this means the LLM needs to output instructions to draw a digital image.

This leaves only a few options for visual, artistic mediums for an LLM:

  • SVG – an XML based plaintext text format for web enabled vector images.
  • HTML – Using the HTML5 canvas tag with Javascript draw commands. It’s well supported in all browsers now.
  • LaTeX – A way to express complex equations which can draw lines but is not very suited for visual work.
  • ASCII – Using text characters to create a visual image by using each character as a “pixel”.

Of these options the only realistic choices are SVG and HTML5 canvas. LaTeX is not really suited for visual work and ASCII is not really suited for actual drawing (it’s great for CLI output or things like comments in web3 smart contracts).

SVGGPT ??? Nope.

SVG turned out to be a little too complex and verbose. It’s a very powerful format but the additional characters it uses with the XML spec + all of the attributes ended up being very difficult to create an image with.

While SVG does work, and it was the first format I tried because it seemed ideal, there were some major issues. Notably limits on output tokens often resulted in partial SVG drawings and without sufficient closing tags for open tags it just wasn’t possible to consistently generate complete images even on a basic level.

HTML5 Canvas GPT ??? Yep.

It turned out that using the 2D context of an HTML5 canvas tag with draw commands in Javascript was the perfect way to draw basics images with an LLM.

Using a very complex prompt that limits the output to only the relevant code I was able to consistently get DrawGPT to output code that would draw images. You are able to see the Javascript draw commands on DrawGPT when you create an image. Give it a try! All the Javacript code for any image is currently open source on the website.

2D canvas context draw commands in Javascript are not really for drawing complex, detailed images. They are more of the standard draw commands you see in most low level visual systems. The commands are things like fill, rect, line, arc, etc. They are not really meant for drawing complex images but they are perfect for drawing basic images.

This is why most of the output of DrawGPT is not detailed imagery like you expect from Stable Diffusion, DALL-E or any of the latent pixel diffusion methods used by other AI art models.

While it would be possible to draw more detailed images using an LLM + Javascript draw commands given the output token limit of the GTP-3 AI calls it is just not feasible for this particular proof of concept.

To note: if the prompt is changed to ask for more detailed images, or more detailed pixel art, then the AI LLM models will attempt to draw more detailed images. But the output will be limited by the output token limit of the GPT-3 API calls.

How Can We Know An LLM Is Drawing Things Correctly?

Once I was able to get the LLM to consistently render images the question became, “Is it drawing things correctly?” There was some difficult at first with more complex scenes or complex objects as it wasn’t clear exactly what the AI was drawing. Are those dots in the sky birds or are they just noise and artifacts like traditional pixel diffusion methods often produce?

It’s easy to see when DALL-E or Stable Diffusion create an image and the tokens are correctly represented but sometimes it’s not so obvious with a simplified image.

One massive advantage of using an LLM for drawing is that you can simply have it tell you what each object is supposed to be. This isn’t really an option with most of the other AI art methods as they are not trained to output text alongside the image perfectly describing each feature or token in the output image. You can always run the output image through CLIP but that does not give insight in to the actual drawing process or specifically what each object should be.

By forcing the output to include relevant code comments in the Javascript (you can see them in the code on the page) I was able to get the LLM to reveal the various objects it was attempting to draw.

I was surprised.

Not only was the LLM (default OpenAI GPT-3 DaVinci) now creating images I was able to verify that the things it was drawing were correct.

DrawGPT Draws Really Well, It Knows What It Is Drawing

It was stunning to see the AI generated images coming out consistently & correctly.

What do I mean by that? For example:

  • Portraits – Things like hair, eyes, nose, ears, mouth are all in the correct places. It draws those things “inside” a circle it will draw for a head and they will be correctly ordered vertically (the eyes are never below the mouth)
  • Landscapes – Mountains, sunsets, birds in the sky, clouds, trees, etc. are all in the correct place. It never tries to put the ground above the sky or have mountains strangely floating in space.
  • Objects – It knows the basic layout of common but complex objects like bicycles, lamps, and many others things. While it cannot draw a fully perfect bicyle the image it renders features the basic elements in the correct places.
  • Animals – It understands the basic layout of animals, including the number of legs, relevant things like ears or fins and attempts to place them correctly. A great sample is the image used for the DrawGPT AI Art Twitter Bot image. You can clearly see it was trying to draw a bird.

Regardless of this used CLIP data the reality is that the LLM is drawing things correctly.

It is not just drawing random things in random places on the image. It does have some issues with relative scaling but it is hardly ever so bad that the image itself is not recognizeable.

It is also drawing things in the correct order. It will draw the ground before the sky, the sky before the clouds, the clouds before the sun, the sun before the mountains, the mountains before the trees, the trees before the birds, etc.

In addition to drawing concrete objects it is also able to draw things like abstract shapes and patterns. It is not perfect but it is able to draw things like circles, squares, triangles, and other basic shapes. It is also able to draw things like stripes, polka dots, and other patterns.

It will use loops, if statements, and other basic programming constructs to draw things like a grid of squares, a pattern of circles, birds in the sky, and fruit on trees.

Sometimes the LLM chooses to express itself with text as well. It is able to use the text commands to label things or make statements within the image itself.

One truly surprising thing was when I send in no subject to draw at all. The AI will just draw something totally random: portraits, fine art, landscapes, and of course it’s all time favorite the Mona Lisa.

It loves to draw the Mona Lisa.

DrawGPT Is Not Perfect

If you use the app you’ll see that yes, the images are very simplistic. They are sometimes difficult to tell visually what you are looking at because it is just a series of boxes and circles.

Portraits will occasionally be unrecognizeable as it will pick similar colors for some things and make the image a mess. I believe that issue could likely be solved very easily with a better model or more specific training data designed to allow better visual responses.

The LLM is not perfect but it is drawing things correctly. If you reference the comments in the code it becomes clear that the concepts and tokens in the image are correct even if it is limited by the simplicity of the medium it has to use.

This is mostly a tradeoff of using simple draw commands in only text to draw images and rarely the issue with the actual output tokens of the AI.

DrawGPT – Adding Some Character + An Impish Twitter Bot

For fun I have the prompt adjust the comments in the code to add a little flavor to the output, often including a humorous take on the prompt or subject matter.

This was important because it gives the images and the output and the entire AI a feeling of being a character that you are interacting with. This is similar to the way people feel they are speaking conversationally with ChatGPT and it incredibly important for interacting with AI.

Seeing as how DrawGPT was able to draw things correctly & provide a little flavor, character, and humor I decided to create a Twitter bot that would allow users to reply to a tweet and have DrawGPT reply with an image. This also allowed me to experiment with incredibly complex input prompts that I would have otherwise not thought of on my own.

If you’d like to use the DrawGPT Twitter bot you can reply to any tweet with “@DrawGPT draw” and it will respond with an image of the tweet you are replying to and include a link to the image on the website so you can see the code & comments as well as share the link.

DrawGPT – A New Way To Create AI Art

DrawGPT will likely never be a commercial hit. The art is too simplistic to appeal to most people and the output tokens are too limited to be useful for most image generation tasks.

At the same time the simplicity of the images, combined with the LLM drawing important features of the subject, often creates a sort of “caricature” of the subject. For example if you have it draw Trump it will almost always try to draw some sort of hair.

It’s a really fun thing & the creativity of the AI LLM and how it draws is pretty mind blowing. It’s also a great way to get a glimpse in to how the AI is thinking.

DrawGPT – The Code & The Images & The Prompt & License

DrawGPT currently uses the stock OpenAI GPT-3 DaVinci model. There are no additional fine tuning or additional training sets added.

At this time I will not be releasing the prompt I am using.

I do list on the website the prompt tokens & the output tokens as returned so users and researchers can get a feeling for what the prompt may be like.

All of the code and images on the website generated by DrawGPT are currently under the CC0 license. This may change some day but the intent is provide an open source & fun project that publicly showcases the concepts for users and AI researchers.

What Is Next For AI Art and DrawGPT?

The front facing portion of every AI that interacts with humans is a language model.

As humans we express ourselves through language. Regardless of if the AI is an LLM or if it is something like Stable Diffusion, Disco, DALL-E, VQGAN, POINT-E, or any other AI we as humans still have to instruct it with language.

At this time I do not have any huge plans for DrawGPT. I may attempt to introduce other LLMs as a sort of litmus test for how visually enabled they are and I will certainly be giving it a spin with GPT-4 when it comes out.

I chose to output the image in 512×512 pixels, the size expected of most img2img inputs for other models so that the outputs can be used as inputs to more complex AI art models so it is fully compatiable with things like Stable Diffusion.

I am extrememly pleased with the way DrawGPT turned out.

I think that I have conceptually proved a few things and hopefully other AI researchers in the future can build with some of the fundamentals & tips & tricks I explored:

  • Visually enable LLMs by including CLIP data in the language training set.
  • LLM must also have sufficient training on the output medium.
  • Use the visual output to correctly identify if the AI and large language model “understands” complex visual concepts.
  • Include code comments or metadata of tokens in the output linked to specific parts of the image to identify if the drawing is “correct”.
  • Give the AI character and flavor to make it fun to interact with.
  • Enable the use of crowdsourced or social inputs to explore complex inputs you would not normally think of yourself.

Did You Write This With AI?

No. The horrendous spelling mistakes and terrible grammar are my own. I’m a programmer, not an English teacher.

Did You Really Not Click the Link Yet?

If you have somehow made it this far in to the article without clicking, now is the time.

Click here to try out DrawGPT and draw your own images with AI and generate art with an AI that only knows written words and has never seen a pixel in its life.

Categories
Announcements

Just When You Thought the Internet was Safe Here’s Trending Domains!

Domain Check Added a Nifty Trending Domains Page

If you caught the earlier post on the recent birth of the impish Domain Check Twitter account then you’re in for a treat; Domain Check has just released a full page dedicated exclusively to trending domain names. This is basically a view for the same data set of trending topics, hashtags, and breaking news that the Twitter bot pulls on to create the tweets. Each day the previous day’s trending domain names are archived so you can aimlessly page back through the domain name ideas and laugh at what a full site dedicated to some of these trends would be like. Its also an easy way to go back and find a trending domain that may have caught your eye earlier from the Domain Check Twitter or the Domain Check Facebook.

Let Other People Be Creative

It can be a bit of a pain in the ass to get creative with a brand name or domain so its always good to have some extra help to get the creative juices flowing. Just paging through a few days worth of amazing Twitter hashtags may very well be enough to kick your brain in high gear or at least get a few chuckles. Its definitely a fun tool, check out the trending domains at Domain Check

Missed the link? Click here for Trending Domain Names at Domain Check!