You Won't Get Left Behind

AI-Indifference

I wouldn’t call myself an AI-skeptic or even an AI-hater, but instead I would say I’m just indifferent. I follow AI advancements on Twitter from a technological standup to a certain point in order to keep up with the state-of-the-art, but I don’t care to ever work in AI. I don’t consider myself too good to work in AI (whatever that would mean) or that I think that I am not replaceable by AI (I totally am), but rather it just doesn’t interest me. Just because the whole world (or media cycle) is focused on AI does not mean that you, me, or anyone has to pay it any mind.

Corporate Usage of Generative AI

Companies are scrambling to capitalize on Generative AI, but why? To say that they are using it? I think that it could be incredibly risky to jump and invest so much money on a nascent technology. If your company is already doing well, I think that it makes sense to begin starting the foundation for using Generative AI, but not going onto using it just yet. If the company hasn’t even gone through a data revolution, then I don’t even know how they could meaningfully use AI in an effective and cost-efficient manner. Using your own data to fine-tune and distill models seems like the best option for corporations, but companies that just blindly buy into the AI hype may end up hurting themselves.

People don’t want to pay for internet services, this is something that has been embedded after years of free services, supplemented only by VC money and advertisements. I am seeing services like Snapchat add AI to their applications, but I don’t see how they are recouping any of that money spent on OpenAI calls. These AI companies also have to charge for there services because training takes an absurd amount of expensive compute, and running a single inference isn’t cheap either.

One way I could maybe see companies justify so quickly making their foray into the AI space would be because trying to hire talent right now and try to retain them, but is this the best way to acquire and retain talent? I think that ultimately companies have not been thinking about long-term strategy for AI because it is all happening so fast and they are worried about missing out and getting behind.

Ignore the Hype

Learn about Generative AI if you want to, not because you feel like you need to.

In some ways, “getting in early” won’t even really help you. Learning how to use Generative AI will only get easier as we understand prompting better. Few shot examples, chain of thought, larger context windows and other things like this will only continue to advance and develop into new techniques as well. You might build expertise, but the field is shifting so much that I would predict that a lot of things you learn will become outdated.

The Gartner Technology Hype Cycle graph shows how the FOMO feelings drive a lot of entrepreneurship, particularly and especially in the Peak of Inflated Expectations.

I feel like everyday I see a new post on Twitter by an AI “influencer” where they will post the output of some new AI tool and boldly ask the question, “Is this the end of X?” to which people in the replies pile on and say, “Absolutely not, this is so bad.” To which I think that both people are kind of right. The AI Influencer is showing off a cool tool that does something somewhat unprecedented, but at the same time they are being disingenuous about the seriousness and influence of the tool they are discussing.

A thread that was particularly sobering and enlightening to me was the following thread about curriculum/pedagogy and about how just because a technology exists, does not mean that we have to engage or integrate with it, it’s not like we don’t have a choice.

This whole thread is worth reading. What I want to press on a bit is this:

Just because a new technology has arrived that disrupts our teaching does not mean the best thing to do is integrate it into our teaching. https://t.co/A0jihnuUjt

— Irina Dumitrescu (@irinibus) May 20, 2023

I think that a lot of times companies and people talk about AI and automation like its an inevitability, but that’s simply not true. Automation is a process that happens via policy and other decisions. I think that the inevitability is born of the capitalist system that we live in, where markets tend towards efficiency, and AI is perhaps the ultimate form of efficiency.

Are transformers the future of AI?

AI does feel genuinely transformative because it is quite novel and a glimpse of something magical, which people want to believe in. Science needs to get clicks. It is a genuine breakthrough in AI. However, that does not mean that we will not see many other breakthroughs.

Transformers are not the future of AI, but they are the present. Transformers are great and are doing pretty well with everything that we throw at them, but just like how we didn’t know how successful transformers would be, we don’t know what the next big breakthrough could be. With that being said, transformers are not done yet, we will continue to see people milk everything we can get out of it for the next five years at least. I think that part of the reason why we feel like AI is advancing at such a fast pace is a combination of a couple of factors. For one, its definitely in part due to the media cycle and social media, the sensationalization of AI is inescapable, mostly in part due to corporate interests I would say. Additionally, with Blockchain and NFTs mostly being played out after a number of crashes, scams, and other debacles, there was a lot of talent and attention that was freed up to be devoted to AI.

I think that something to note is that we are seeing lots of pre-prints, but how many of these are peer reviewed and get published in journals? People are researching these things because they know that it will get read. Other areas have promise too, like Liquid Neural Networks, but you don’t see many people talking about that.

I think that transformers as a whole could become the next string theory. It is productive and does a lot, but eventually will hit a wall, and then the evangelists will keep on trying to make it work, but slowly everyone else will move on. Is this a likely thing to happen? I don’t really have enough expertise to say, but I think that it is certainly something interesting to think about.

• Learn more about the rise and fall of String Theory with this YouTube video