TL;DR
Here are is my ranked list of suggestions, based on my own experiences and use-cases:
- ChatGPT Plus is probably the most versatile, full-featured, multi-modal language model available in early 2024. The mobile app and ability to interact with voice is very good. But it’s expensive compared to other options.
- If you’re going to pay 20 GBP / month, then Gemini Advanced is proving to be a very impressive competitor, and seems to be GPT-4 equivalent in terms of the quality of it’s outputs. The interface is very nice, and it has some cool features built-in. If Gemini enabled a wider range of document uploads directly from the chat window, I’d probably put it at the top of this list. And if most of your work takes place in the Google ecosystem, you should consider Gemini (free or paid) over ChatGPT.
- Copilot is increasingly capable and the integration with the rest of the Microsoft ecosystem is compelling, especially if you have access through an institutional license. If most of your work is through Microsoft products, this is a good option.
- I think Claude is the best free GPT-3.5 equivalent model, allowing a wide range of document uploads, with very good responses to the kinds of prompts I use. Even with paid tiers available to me, I find myself using Claude often.
- Perplexity is great if you need more confidence in the accuracy of responses. I like that it will often follow up with questions to help refine it’s output.
Over the past year or so, I’ve been experimenting with a few different language models and image generators. Over time, I narrowed in on Claude, and wrote about my preference for using it over other options. A couple of months ago, I was recommending Claude for most people, mostly because it was a GPT-3.5 level model (i.e. it was kind-of equivalent to the free version of ChatGPT) and I thought it’s outputs were ‘better’. Side note: ‘better’ in this context is entirely subjective and almost always means ‘better for me, in specific contexts’.
Early 2024 updates
Over the past couples of months there have been a few developments that have led me to revise the advice I tend to give people who ask what I use.
- Google released Gemini, their most advanced language model, and I’ve found that it is very good.
- I signed up for ChatGPT Plus (the paid version) for a month and used it almost exclusively on the desktop and mobile app.
- My institution enabled Copilot (previously, BingChat for Enterprise), which uses the GPT-4 model from OpenAI, with some modifications from Microsoft.
- Perplexity seems to keep getting better and better, especially for use-cases where I prefer accuracy over creativity. I don’t talk about it much here, but Perplexity is another model I return to fairly regularly.
- Open source models are getting better, quickly.
Based on my experiences with these new options available to me, here are some updated thoughts on what models to use.
If you can afford it, ChatGPT Plus is outstanding. I immediately felt that it was significantly better than GPT-3.5 (the free version). It’s outputs are excellent and the quality of the back-and-forth exchanges I’ve had with it have been outstanding. The integration with DALL-E for image generation is also great and I used this feature a lot more often than I expected. My use case is typically generating a nice cover image for slides for lectures and presentations. I really like the Custom Instructions, which are also available in the free version. Having said that, I didn’t think it was worthwhile to keep up my monthly subscription.
There are 2 main reasons that I stopped paying for ChatGPT Plus:
- Gemini from Google is a significant improvement over PaLM 2, which means that Google’s chatbot is now as good (IMO) as ChatGPT Plus (it’s not just me who thinks this; see here and here, for example). Even if you think that ChatGPT Plus is better than Gemini, I think you’d find it hard to argue that it’s 20-Pounds-a-month better. Granted, Plus makes it easier to upload documents (Gemini requires that you use extensions to connect to documents in Drive) and it has other features (e.g. custom GPTs) that may not even be on Google’s roadmap. However, Google seems to release product updates more frequently than OpenAI (Gemini 1.5 is coming out soon), so I expect Gemini to keep getting better, more quickly.
- Microsoft has upgraded the institutional licenses for universities so that Copilot is included by default. And Copilot is powered by GPT-4, which is the same model behind ChatGPT Plus. I’ve never been a fan of the Edge browser, but the integration of Copilot as a sidebar works very well and the ability to browse PDFs in Edge and have Copilot interact with it is a nice feature. I’ve found myself using Edge more and more often at work, purely because of the ability to quickly open the Copilot sidebar. I also like the images it generates and use this feature often.
All that’s to say that I can get GPT-4 level outputs without paying for ChatGPT Plus.
Gemini
In particular, the release of Gemini Advanced has made the paid option of ChatGPT much less compelling. I signed up for Advanced as soon as it was released and will be using it for the 2 months of free access you get when you sign up. I’ve been impressed with the quality of responses from Gemini Advanced, having used it alongside ChatGPT Plus and testing them with the same prompts.
I’m excited about the potential of Gemini to interact with other apps in the Google ecosystem, especially YouTube. Extensions have been available in Bard / Gemini for ages but because it’s been powered by PaLM (which wasn’t as good as either GPT-3.5 or Claude), I never really used them.
For me, the biggest drawback of Gemini is that you can only upload images directly from the prompt window, whereas ChatGPT Plus and the free version of Claude allow for a much wider range of documents to be uploaded directly. With Gemini, I need to first upload documents to Drive, and then use the relevant extension to work with it. Because so much of my work includes interrogating text-based documents, and all of my work-work (as opposed to personal-work) takes place in the Microsoft ecosystem, this extra step is a hassle. However, the quality of the interactions is such that, for now, the hassle is worth it.
Moving forward, I may decide that the combination of the free versions of Gemini and Claude is good enough compared to 20 Pounds month for Gemini Advanced (the model I use most often at the moment).
Claude remains an excellent model, with most of it’s features available for free. I like Anthropic’s constitutional AI policy, as well as the fact that you can upload documents directly in the prompt window. In general, I find that it’s responses have less ‘fluff’ around the edges, consistently giving me what I’m looking for. And so far, I’ve also preferred it’s ability to take on personas, whereas ChatGPT and Gemini have sometimes struggled to understand what I’m looking for.
Summary
In summary, I find myself using Gemini Advanced as my go-to model for most things. It has almost entirely replaced ChatGPT Plus, to the extent that I no longer think it’s worth paying for even though I still think ChatGPT Plus is the more advanced total package. And when my 2-month free trial of Gemini Advanced runs out, I think I’ll keep using the free version of Gemini anyway. Because alongside it I have institutional access to Copilot, as well as Claude. And the combination of those features means that I find it very hard to make the argument that ChatGPT Plus is worth 20 Pounds a month.
Note: I also recently came across OpenChat, another GPT-3.5 level model, which is open-source. Almost every piece of software I use is open-source, so this appeals to me on a philosophical level. In Dec 2023, OpenChat outperformed ChatGPT (free version) on some benchmarks, so it may be reasonable to assume that it’s also Claude-equivalent in it’s outputs. OpenChat has some interesting features that I haven’t seen anywhere else. For example, the ability to save chats and prompts into folders makes it possible to organise your interactions with more granularity than simply having a single list. If you’ve used generative AI a lot, you soon realise that it can be difficult to revisit older conversations. Being able to save them into folders could be a nice way to collect noteworthy interactions.