With AI, the Future of Augmented Reality is in Your EarsCrazy Stupid Tech · November 9, 2024 |
By Om Malik Dennis Crowley has built his career at the intersection of emerging technologies and human behavior. Twenty years ago, as a 25-year-old inspired by Harry Potter's Marauder's Map, he created Dodgeball—turning text messaging into a way for friends to find each other in the city. When the iPhone emerged, he launched Foursquare, riding the convergence of GPS, apps, and our growing comfort with social networks to transform how we interact with places around us. Google bought Dodgeball in 2005 and eventually shut it down in 2009. Dennis left Google to start Foursquare with his co-founder Naveen Selvadurai. Foursquare rose like a meteor, and much of its core functionality was subsumed by Facebook. Mark Zuckerberg has no qualms about copying others’ ideas and then presenting them as his own. Foursquare, sadly, has since lost its preeminence in the location sweepstakes. After giving up his role as the chief executive, Crowley ran the R&D Labs at Foursquare and went off to try a few new ideas. One of them, Hopscotch Labs, has become his latest startup, he has co-founded with Max Sklar and Alejandro Fragoso. With Hopscotch Labs, Crowley is exploring what happens when ubiquitous AirPods meet artificial intelligence. Like his previous ventures, this new project arrives at a moment when certain technologies have become commonplace while others are just becoming possible. The new company is building a service that leverages the enormous corpus of data in large AI systems and newly ubiquitous headphones. Then it’s adding the capabilities of a smartphone to surface relevant local information as and when a person walks by a location, such as a restaurant, a cocktail bar, or even a street corner. It feels like a panacea for information overload. Instead of looking into a browser or constantly peeking into a phone, the information comes into our ears or to our other devices, such as glasses. I have followed Crowley’s career since 2000. He has a preternatural ability to see the future, especially when it comes to location and information architecture. With Dodgeball, he correctly predicted the confluence of machines responding to our text messages. Foursquare correctly predicted the importance of geolocation as an everyday axis of search and discovery. And that is precisely why I am interested in what Crowley has to say about our future. In my conversation with Crowley, we discuss his latest startups, convergence of new technologies, and what it means for the future. We get into a conversation about AI, our children, and entrepreneurship. He talks about attempting to make sense of it all while staying true to his original mission: building things that help people better experience the world around them. -- OM: So I don't know if you remember, but you told me about this idea you were working on in 2019 called MarsBot. Is Hopscotch Labs a variation of that idea? Dennis: Yeah, but we made it and were going to launch it at South by Southwest, and then the pandemic hit. So we started focusing on different things. My team worked on COVID data tools. We did things with billboards in New York City, tools that told you when was the best time to go to the supermarket. Later, we wound down the Labs team. After I left, I couldn’t stop thinking about that idea. Last year, I reached out to Max (Sklar), who I used to work with at Foursquare, and told him that it was a fun project and deserved to exist. We started off trying to resurrect the project from scratch, but it was hard because the API access (to Foursquare data) we had as outsiders was much different from the API access we had as insiders when working for Foursquare. So we ended up building a “sequel” to Marsbot, much like how Foursquare itself was a sequel to Dodgeball. OM: Then what? Dennis: We thought we should use AI tools to do it, and it turns out that works pretty well. Let me tell you about the product. It's an iPhone app.There's not much to do in the app, there aren't a lot of buttons to press. Once it's installed and you set the permissions, every time you put on your headphones — your AirPods or any other headphones, it doesn't matter—it chimes and says, "Beebop's been activated." (Beebop is the first project from the company, Hopscotch Labs.) Then as you walk around the city, it will tell you things about certain places. Eventually, if I walk by a place where my friend was, it tells me that Alex was here two days ago. If I walk by a place and someone's inside, it tells me that Max is inside that place. A lot of it is still under development. Eventually, people leave a comment at a place. Imagine Twitter. It is as if you leave a tweet and you stick it in the ground. When you walk over it, you hear it. OM: Why are you using AirPods as a way to share this information? Didn’t you once say, audio is the poor man’s augmented reality? Dennis: The big idea here is that the AR layer can be beyond the visual layer. Essentially, augmented reality doesn't have to be just through glasses; it can be in the ear. If you went back to the blog post that we wrote at Foursquare about Marsbot, one of the headlines was about using audio as the poor man's augmented reality. I'm not waiting for the glasses. I'm going to make something today. There's more pervasive use of headphones now than there was in 2019. You probably see it all the time in San Francisco; 40 percent of the people you walk by in New York have AirPods on. [Research from Statista shows that 34.4% of U.S. adults use Apple headphones, with an additional 15.3% using Apple-owned Beats products. Another survey found that 62% of Americans between 18 to 24 years old own AirPods. -- Ed.] Some of them are listening to music, podcasts, or on a phone call. Occasionally, our software will chime in and tell them something. We are experimenting and starting to play with stuff, like speaking and broadcasting that audio to people in the neighborhood, people in this room, or people across town. OM: We're in a post-social era. The massively scalable social environments are not that effective. What you're describing is essentially a lot more intimate — only friends can hear your recommendations or messages, much like you did in Foursquare. The collective intelligence shared by everyone was shared with others through the app. Dennis: Right now, we're building this without a social graph, and we're going to try to use AI filtering tools to make it less about who you follow and more about things that are relevant to you. I don't know if that's going to work, but instead of spending our cycles trying to build something for friends — I don't know if people have the bandwidth to do that — everyone is going to hear everything. It's our job to act as the filter. OM: So you will use algorithmic mediation? Sort of what Facebook is doing with Threads? [You get two streams: an algorithmically picked “For You” feed that Facebook decides what you need to see, and then a feed for those people you “follow,” which is in a separate tab.] Dennis: We are using geolocation as a filter. If it's in New York, in a specific neighborhood, and someone was here in the same context as me and left a comment or information, then it could be surfaced. OM: Last time you and I talked about the future of mapping, you mentioned building Harry Potter's map. [In the Harry Potter series, the Marauder's Map is a magical document that shows details such as every classroom, hallway, and secret passage from Hogwarts School of Witchcraft and Wizardry, and at the same time tracks the location of every person in the castle and grounds, including ghosts, using named dots.] If you look around, Google Maps is pretty close to that idea. Not visually, but in terms of data richness, they've actually done a good job. Dennis: I think Snap Maps is probably closest to it. [Snap Maps is a map application launched by Snap in 2017 that allows you to follow your friends on a map.-- Ed.] But I don't use it. My friends don't use it. Swarm, a location app launched by Foursquare, took a swing at it. Maybe Foursquare circa 2013 was a pretty good version of it. Maybe it's hard for me personally to say out loud that I'm going to do the same thing again. As an entrepreneur, you're supposed to look for different opportunities. There's this pressure that you should be doing something else. I just like to build the same thing. I want to build the thing I want to use. At the end of the day, anything I've ever built that's been successful has always started with what's the thing that Dennis wants and Dennis can give to his ten friends, and then maybe they'll use it, and then maybe those friends like it too. OM: When I first talked to you about Foursquare back in the day, you had this wonderful analogy — you said you wanted to build Harry Potter’s Map. What’s the analogy for this? City Guide? Dennis: This is not a City Guide for AirPods. It is so much different than that. First, I thought it was an AI copilot for walking around. That sounded stupid. An AI assistant for cities? That's not it either. And then one of the guys I'm working with said, it's Marauder's Map. That's it. It's the Marauder's Map for AirPods. OM: Clubhouse founder Paul Davison, with Highlight, tried this idea of surfacing ambient information and using collective information. Why do you think now is the time? What is the convergence of technologies, trends, and behaviors that make you think now is the time to start experimenting with this idea? Dennis: I came across Marauder's Map when I was 25 years old. I said, I'm gonna make Harry Potter's Marauder's Map for text messaging. That led to Dodgeball. The reason it worked was because cell phones were newly ubiquitous, and text messaging was newly ubiquitous. The idea that you could send a text to a machine, and a machine would parse it and send you a response — that was new. Foursquare was made two years after the iPhone came out. The iPhone was newly ubiquitous. People already knew how to use GPS and apps. People understood social networks. I didn’t have to explain it to them. People knew how to download an app. I didn’t have to do as much work as I had to when explaining Dodgeball to people. Today, what's ubiquitous? Everyone has headphones. What's newly possible? AI, right? You look at the intersection of how things are coming together. It's about putting these two things together and just trying to make something. Will this thing work? I don't know. But I didn't think Foursquare was going to work, and I didn't think Dodgeball was going to work either. OM: From my own perspective, when I looked at Dodgeball, it seemed like a programmable interface between text and machines.That reality is now pretty commonplace 20 years later. We conduct so many transactions via SMS that we don’t even think about it. When Foursquare came out, I saw the convergence of location, sensors, and emergence of a new class of applications. It’s now a commonplace experience — we use that idea for ordering food or rides. When I look at what you're doing with this new company, it reinforces my own thinking about the idea of a world where ambient information is all-pervasive. Whether it is through glasses, through the ears, through the phone, or a device that doesn't look like a phone, it is likely going to be conventional thinking 10 years from now. Dennis: Being in this space for so long, when I walk into a movie theater, why can't anyone tell me which of my friends are here? How is that still not a solved problem? I am always thinking about what's the most unobtrusive way to solve that problem. Why? Because I'm over this idea of building something for your phone that you have to scroll through. I don't want to do that anymore. We tried notifications. We did a lot of work with notifications at Foursquare. So that's why we're trying audio. Anyway, back to your point. Dodgeball was about phones and texting computers. But at the end of the day, you magically made people show up at the bar, and that was a f***ing really good magic trick. Foursquare was possible because of sensors, text, and GPS. But then, people fought over being the mayor of a coffee shop. OM: You are talking about me, fighting over the well-chronicled mayorship of Sightglass Coffee, right? We laugh! Dennis: That's a silly thing, but that's what people fell in love with. This time around, I don't know what that thing is, but if you ask me in a year, I would say there's a high likelihood that we will find that thing which brings out the emotion. I pitched a couple of VCs on this, and they're like, why is it going to work? I'm like, because we're going to make people fall in love with it. How do you do that? I'm like, I don't know. We'll just keep making stuff until we find something that people fall in love with. OM: I understand the emotion part, but one of the reasons why I am always interested in what you do is because of the pattern. Dodgeball and Foursquare both predicted the future and, more importantly, a bigger shift in societal behaviors. Today, we look up everything using maps. Everyone does it all the time. When I saw your thing, I wondered if this is what we are all going to be doing in 10 years from now. Dennis: Will it work? I don't know. No one knows what's going to work. OM: Tell me more about you as a founder. How have you changed? We've talked about three different phases of your life. When you were a complete rookie having just started Dodgeball. We talked when you were facing some tough times at Foursquare. Now you're starting this. How have you evolved both as a person and as an entrepreneur? Dennis: I feel much more like my authentic self than ever before. And I think it's because my priorities are straight now. I don't have a lot to say or prove, but I still have a lot to build. I want my kids to realize that there are people in the world who make stuff and that their dad makes stuff. OM: So back to building? Dennis: It's been a long time since I built something. What's different now? AI is what's different. You know what's different? Everyone's on their phones all the time.Back in my day, we made stuff for the streets. You would use your phone, you'd put it away, and stuff would happen. Where are the people making that stuff anymore? All we make is stuff that makes you look at the screen. Let's make stuff that gets you out in the world, where you're not glued to the devices. You're out there doing stuff. Big companies out there aren't going to make stuff like this. OM: Did you lose that building feeling? Or is it like you've regained it? Dennis: It's your art. This idea of art isn't just painting and music. It's the way any individual expresses themselves doing something—snowboarding, skateboarding, dancing, whatever. I make weird stuff, right? I'm at peace with that. I want to make another thing that people love, in the same way an artist makes things to evoke response and reaction. OM: What role do you think AI will play in our day-to-day life in the future? You're clearly building an app that uses AI, so you must have some idea of how it is going to help reduce friction in how we consume information. Do you have any bigger picture thoughts about it? Dennis: There is a whole generation of builders who are using AI. Scarlett Johansson’s character in “Her” is their North Star. That’s where it will probably end up — a personal assistant. That’s a little bit of screen and a little bit of voice and speech that is proactive and can help you with everything every day. I don’t know what the alternative to that is. So everyone has a personal assistant with them 24 hours a day, consuming everything that you consume, and also seeing the world through some kind of AirPods with a camera or glasses with a camera or something. It might be seven or eight years, not five years. I would be lying if I said I'm not super excited about that future. At the same time, if that future is going to be handed to us, I feel like we can build a slightly different version of it. And that's what we should do. I feel like I would kick myself if I just sat around waiting for someone to build that instead of trying to make a version of it. OM: When I see “Her,” I see two aspects to the movie. One is very dystopian, and the other is that it is a love story. It was a story of someone who was lonely and looked for a connection. Silicon Valley has totally missed that part. Valley’s missing empathy gene is reinforced in how everyone talks about agents and personal agents. So what is that more optimistic, more humanistic future we can build from an AI perspective? Dennis: I don't think regular people are prepared to think about the idea that maybe a third of your friends are digital friends. This is something Chelsea (his wife) and I talk about a lot, especially when we talk about the kids. These are really toys for kids. AIs that are meant to grow up with them. If you want to go back to Harry Potter, right? There were seven books, and every year a book would come out, and every year the characters would age. Those are characters that grew up with their readers. The characters were changing in the same way that the readers were changing. And that was a really interesting and beautiful thing, especially for it to be a mass phenomenon. Now you're talking about AI Pokémon, or an AI creature, whatever it is, that is going to grow up with the kids. AI is going to be my video game companion. I'm not going to play Fortnite with my cousin; I'm going to play Fortnite with my AI friend. That's weird today, but I think that's just going to be normal. Will AI friends be better friends than real friends? Can you get what you need out of an AI relationship that you get out of an in-person relationship? I don't know. People certainly get it through online dating. They get it through long-distance relationships. So, you know, it does work. OM: When you started Dodgeball, there was a certain kind of information density. Not as much. With Foursquare, that information density had gone up quite a bit. And perhaps that is why we needed a way to visualize it. Foursquare was one of the many applications that came around that time, which looked to arrange information in a consumable fashion. Fast forward to today, that information density has just exploded to a point that it's humanly not possible to deal with it. Whether it is Facebook, Threads, Twitter, or Instagram, these platforms are trying to provide a curated, algorithmic version of information to reduce its density into a consumable form. Do you think AI becomes a better way to sift through this massive amount of data? Dennis: In a way, all the content you'd ever need already exists. But as a developer, I can't pull that out of Instagram. Only Instagram can do that. I can't pull it out of TikTok. Only TikTok can do that. I can't pull it out of Twitter. Only Twitter can do it. Threads doesn't have an API. You have to look at all these proprietary LLMs that have absorbed all the data. ChatGPT has scraped everything from Yelp and Foursquare, from Reddit, and from everyone else. It's all there. All the stuff we spent ten years building, they just took. It's already there. You ask it (OpenAI) a question — say a cocktail place around here, it tells you. What is not solved? The ability to figure out the thing that Dennis might be interested in right now and surface it to him as if I were his best friend walking next to him. So you have to create a service that generates some kind of content. You need to apply your own AI to filter that content and combine it with other pieces. That's what we're doing. We talked about AI friends and Scarlett Johansson — that's the path we're on. OM: Are you using OpenAI? Dennis: We use OpenAI, we use Claude, we bounce back and forth. And we're doing a lot of experiments now. Eventually, we want to figure out how to turn people walking down the street into reporters. You know what Waze does for driving, do that for neighborhoods. How do you annotate the world in an ephemeral way and then share that back with people? And do it without checking in. OM: The challenge for you would be to reconcile privacy and how people think about it. But then again, privacy as a concept is different for a new generation of internet users. Dennis: If anything, people are really buttoned up about privacy in a way that they weren't before. They care much more. I'm not going to give you location permissions. That's the thing that we have to overcome. OM: This is all very cool. I'm excited for you. My two previous conversations with Dennis Crowley:
|