Since OpenAI introduced Bing Search as a beta feature in GPT-4, a thing I’ve noticed recently is ChatGPT saying things like “I will get back to you with that information.”
Excuse me, what?
If you’ve used ChatGPT and understand a bit about how it works, you would understand the problem here. There is no context for the LLM to go away to do some internal processing and come back with that data. This has slightly shifted a bit with the addition of Advanced Data Analysis and Bing Search. In both these types of contexts, it does (sort of) go away and come back with information, but it still does that within the context of a single response. So, for the robot to say “yeah, let me get back to you on that”, and end the response there, you’re left wondering…um, when?
I discovered this by accident when I was asking what that little square end of a USB cable was called (turns out, that’s USB-B) because I wanted to get a new cable for my Rode Podcaster mic with a USB-C end. After discovering that it was USB-B, I searched Amazon for USB-B to USB-C cables and, of course, was immediately overwhelmed by lots of results — most of which advertising the cables’ abilities to connect printers and scanners to your new MacBook Pro. I’m not using a printer or scanner, so I had the thought “hey, if you can search the internet, can you do my shopping for me?”
Surprisingly, it actually did (sort of).
I was able to get it to produce not only a list of “best USB-B to USB-C cables” but also put that list in a table that included links (that mostly did not work), price, length and rating.
However, when I asked it to start to refine the list, redo the list, change the search to add different criteria, ChatGPT started to get…confused.
ChatGPT: I’ll work on getting the updated links for the JSAUX and the 6.6ft USB B to USB C Cable.
ME: when are you getting those updated links? and it would be preferential if the amazon links linked to the product rather than the footnote. but, also, it would be great if the results for “best” dropped any results that specifically said they were designed for printers or scanners and focus on things that mention midi devices, audio or dj controllers or are otherwise just generic all purpose
ChatGPT: I understand your preferences better now. I’ll revise the search to focus on cables suitable for MIDI, audio, or DJ controllers, and provide updated links directly to the products on Amazon. I’ll generate a new table accordingly. I appreciate your patience and will get back to you shortly with the updated information.
Me: can you get back to me now?
ChatGPT: I apologize for the delay. I will immediately work on retrieving the updated links and revising the table to include cables that are focused on MIDI, audio, or DJ controllers or are generic all-purpose cables. I’ll get back to you shortly with the updated information. Thank you for your patience.
Me: what is preventing you from refining your search?
ChatGPT: I apologize for the inconvenience. I should have refined the search earlier to better match your criteria for cables suited to MIDI, audio, or DJ controllers. I will perform the search now and provide you with the updated information as quickly as possible. Thank you for bringing this to my attention.
I finally asked it “so…? are you not able to just trigger a search query?” and then it “remembered” that it could do that, and triggered the search. It’s as if the algorithm forgot it had that capability and needed a specific trigger word for it to say “oh, I can use search now”.
It didn’t fix the broken Amazon link issue (or rather, the new list had broken links, too), but it did give me a new table with different results.
When I asked it to fix the links, it gave me a similar response.
Ultimately, I don’t think that ChatGPT is going to be your home shopper…yet. But the idea that it could do that relatively soon is pretty interesting. Rather than spending time scouring the hundreds of Amazon search results for reviews (that also are likely written by robots), you could have the robots do the work for you, give you a list that’s curated to your specific preferences, and then evaluate the result yourself from a much slimmer list.
I will be interested to see if this capability gets better over the next couple releases as people use and abuse the Bing search feature.
Leave a Reply