Tag: DeepMind

Ten years ago, Pear VC, then a tiny new venture firm, operated out of a nondescript office in Palo Alto that was enlivened by bright, computer-themed art. Last week, the outfit — which closed its largest fund to date in May — quietly inked a deal to sublease 30,000 square feet of “Class A” office space in San Francisco’s Mission Bay neighborhood from the file-storage giant Dropbox.

It’s among a number of fast-growing outfits taking up more space in San Francisco as an earlier generation of companies shrinks its physical footprint.

As the San Francisco Chronicle first reported last week, ChatGPT creator OpenAI just subleased two buildings totaling a collective 486,600 square feet from Uber. The ride-share giant, which originally leased a grouping of four buildings down the street from Dropbox and will continue to occupy two of these, told the paper it is “right-sizing.”

A rival to OpenAI — Anthropic — also just reportedly closed a sizable subleasing deal. Its plan: to take over the entire 250,000-square-foot building in downtown San Francisco that was previously Slack’s headquarters.

Salesforce, which acquired Slack in 2021, is an investor in Anthropic. Meanwhile, Pear VC co-founder Pejman Nozad wrote one of the first small checks to Dropbox when he was still relatively new to the U.S. from Iran and selling Persian rugs to Silicon Valley bigwigs.

Such subleases don’t necessarily begin with hand-shake deals, however. Asked if Nozad zeroed in on Pear’s new space owing to his connection to Dropbox, he scoffs. The office — which has room for more than 200 desks, features more than 20 conference and call rooms, and has dedicated event space to host talks — “was a business deal for them,” says Nozad. “The founders were not involved. As you know, I sold rugs for 17 years, so I have some skills in negotiation,” he adds with a laugh.

Certainly, it’s a good time to strike a subleasing deal if you’re a well-funded company on the rise. According to Colin Yasukochi, an executive director at the commercial real estate services firm CBRE, subleases in prime areas like Mission Bay and the city’s Financial District currently range from $60 to $80 per square foot. The higher the floor and the more plentiful the amenities, the higher the price. For startups willing to sublease space with less than five years left on the lessee’s contract, the better the terms (as they’ll need to lease again somewhere else in the not-too-distant future). In comparison, office lease rates passed the $75 per square foot mark in September 2019 before the pandemic turned the city upside down.

There’s no shortage of options right now. San Francisco’s commercial buildings are currently 35% vacant, and there are still more tenants flowing out the door than entering them.

Dropbox originally leased the entire 750,000-square-foot space in the building it currently occupies, but it never filled it up entirely and after COVID struck, it began more aggressively whittling down its use. It paid $32 million in late 2021 to terminate part of its 15-year lease; before newly subleasing space to Pear VC, it separately subleased roughly 200,000 square feet to two different life sciences companies: Vir Biotechnology and BridgeBio. It’s still less than half full.

This week, Adobe listed half its leased footprint in San Francisco’s Showplace Square neighborhood and is now looking to sublease 156,000 square feet across three floors of one of the buildings it used to occupy.

But a tipping point is seemingly in sight. There was “negative net absorption” of 1.85 million square feet in San Francisco in the third quarter of this year, according to CBRE data; at the same time, market demand reached 5.2 million square feet, which is the highest increase since the first quarter of 2020.

Much of that shift can be traced to companies like OpenAI, suggests Yasukochi, who says that a new spate of outfits is starting to set up shop, enticed by the opportunity to rent sleeker space for the same or better prices than was possible several years ago for less finished locations, and in more central areas of the city. “It’s a huge opportunity for companies that are trying to bring back their employees,” says Yasukochi. (OpenAI CEO Sam Altman has long said he thinks companies are more effective when employees convene in person.)

Indeed, Yasukochi anticipates that if the economy improves in the second half of new year and interest rates come down, tech outfits in particular will be positioned to recover faster — and pull the city along with them. “Many tech companies were quick to cut excess employees, along with real estate and other costs,” says Yasukochi. He also says that while tech outfits are typically “early to cut back, they’re also early to grow. I don’t see any other industry that generates the volume of growth that tech can.”

Worth noting: Yasukochi does not think those tech companies will necessarily be growing in San Francisco’s Hayes Valley. Though the small shop-studded neighborhood has led a resurgence of interest in San Francisco this year and eagerly embraced the moniker “Cerebral Valley,” owing to its concentration of AI communities, most of those teams, he observes, are “meeting in restaurants and bars and working out of their apartments.”

The reality, Yasukochi continues, is “there isn’t a lot of office space there.”

Pictured above: 1800 Owens Street in San Francisco, which is the site of Dropbox’s headquarters and now, Pear VC’s San Francisco office, too.

[A version of this piece first appeared in TechCrunch’s robotics newsletter, Actuator. Subscribe here.]

Earlier this month, Google’s DeepMind team debuted Open X-Embodiment, a database of robotics functionality created in collaboration with 33 research institutes. The researchers involved compared the system to ImageNet, the landmark database founded in 2009 that is now home to more than 14 million images.

“Just as ImageNet propelled computer vision research, we believe Open X-Embodiment can do the same to advance robotics,” researchers Quan Vuong and Pannag Sanketi noted at the time. “Building a dataset of diverse robot demonstrations is the key step to training a generalist model that can control many different types of robots, follow diverse instructions, perform basic reasoning about complex tasks and generalize effectively.”

At the time of its announcement, Open X-Embodiment contained 500+ skills and 150,000 tasks gathered from 22 robot embodiments. Not quite ImageNet numbers, but it’s a good start. DeepMind then trained its RT-1-X model on the data and used it to train robots in other labs, reporting a 50% success rate compared to the in-house methods the teams had developed.

I’ve probably repeated this dozens of times in these pages, but it truly is an exciting time for robotic learning. I’ve talked to so many teams approaching the problem from different angles with ever-increasing efficacy. The reign of the bespoke robot is far from over, but it certainly feels as though we’re catching glimpses of a world where the general-purpose robot is a distinct possibility.

Simulation will undoubtedly be a big part of the equation, along with AI (including the generative variety). It still feels like some firms have put the horse before the cart here when it comes to building hardware for general tasks, but a few years down the road, who knows?

Vincent Vanhoucke is someone I’ve been trying to pin down for a bit. If I was available, he wasn’t. Ships in the night and all that. Thankfully, we were finally able to make it work toward the end of last week.

Vanhoucke is new to the role of Google DeepMind’s head of robotics, having stepped into the role back in May. He has, however, been kicking around the company for more than 16 years, most recently serving as a distinguished scientist for Google AI Robotics. All told, he may well be the best possible person to talk to about Google’s robotic ambitions and how it got here.

At what point in DeepMind’s history did the robotics team develop?

I was originally not on the DeepMind side of the fence. I was part of Google Research. We recently merged with the DeepMind efforts. So, in some sense, my involvement with DeepMind is extremely recent. But there is a longer history of robotics research happening at Google DeepMind. It started from the increasing view that perception technology was becoming really, really good.

A lot of the computer vision, audio processing, and all that stuff was really turning the corner and becoming almost human level. We starting to ask ourselves, “Okay, assuming that this continues over the next few years, what are the consequences of that?” One of clear consequence was that suddenly having robotics in a real-world environment was going to be a real possibility. Being able to actually evolve and perform tasks in an everyday environment was entirely predicated on having really, really strong perception. I was initially working on general AI and computer vision. I also worked on speech recognition in the past. I saw the writing on the wall and decided to pivot toward using robotics as the next stage of our research.

My understanding is that a lot of the Everyday Robots team ended up on this team. Google’s history with robotics dates back significantly farther. It’s been 10 yeas since Alphabet made all of those acquisitions [Boston Dynamics, etc.]. It seems like a lot of people from those companies have populated Google’s existing robotics team.

There’s a significant fraction of the team that came through those acquisitions. It was before my time — I was really involved in computer vision and speech recognition, but we still have a lot of those folks. More and more, we came to the conclusion that the entire robotics problem was subsumed by the general AI problem. Really solving the intelligence part was the key enabler of any meaningful process in real-world robotics. We shifted a lot of our efforts toward solving that perception, understanding and controlling in the context of general AI was going to be the meaty problem to solve.

It seemed like a lot of the work that Everyday Robots was doing touched on general AI or generative AI. Is the work that team was doing being carried over to the DeepMind robotics team?

We had been collaborating with Everyday Robots for, I want to say, seven years already. Even though we were two separate teams, we have very, very deep connections. In fact, one of the things that prompted us to really start looking into robotics at the time was a collaboration that was a bit of a skunkworks project with the Everyday Robots team, where they happened to have a number of robot arms lying around that had been discontinued. They were one generation of arms that had led to a new generation, and they were just lying around, doing nothing.

We decided it would be fun to pick up those arms, put them all in a room and have them practice and learn how to grasp objects. The very notion of learning a grasping problem was not in the zeitgeist at the time. The idea of using machine learning and perception as the way to control robotic grasping was not something that had been explored. When the arms succeeded, we gave them a reward, and when they failed, we give them a thumbs-down.

For the first time, we used machine learning and essentially solved this problem of generalized grasping, using machine learning and AI. That was a lightbulb moment at the time. There really was something new there. That triggered both the investigations with Everyday Robots around focusing on machine learning as a way to control those robots. And also, on the research side, pushing a lot more robotics as an interesting problem to apply all of the deep learning AI techniques that we’ve been able to work so well into other areas.

Was Everyday Robots absorbed by your team?

A fraction of the team was absorbed by my team. We inherited their robots and still use them. To date, we’re continuing to develop the technology that they really pioneered and were working on. The entire impetus lives on with a slightly different focus than what was originally envisioned by the team. We’re really focusing on the intelligence piece a lot more than the robot building.

You mentioned that the team moved into the Alphabet X offices. Is there something deeper there, as far as cross-team collaboration and sharing resources?

It’s a very pragmatic decision. They have good Wi-Fi, good power, lots of space.

I would hope all the Google buildings would have good Wi-Fi.

You’d hope so, right? But it was a very pedestrian decision of us moving in here. I have to say, a lot of the decision was they have a good café here. Our previous office had not so good food, and people were starting to complain. There is no hidden agenda there. We like working closely with the rest of X. I think there’s a lot of synergies there. They have really talented roboticists working on a number of projects. We have collaborations with Intrinsic that we like to nurture. It makes a lot of sense for us to be here, and it’s a beautiful building.

There’s a bit of overlap with Intrinsic, in terms of what they’re doing with their platform — things like no-code robotics and robotics learning. They overlap with general and generative AI.

It’s interesting how robotics has evolved from every corner being very bespoke and taking on a very different set of expertise and skills. To a large extent, the journey we’re on is to try and make general-purpose robotics happen, whether it’s applied to an industrial setting or more of a home setting. The principles behind it, driven by a very strong AI core, are very similar. We’re really pushing the envelope in trying to explore how we can support as broad an application space as possible. That’s new and exciting. It’s very greenfield. There’s lots to explore in the space.

I like to ask people how far off they think we are from something we can reasonably call general-purpose robotics.

There is a slight nuance with the definition of general-purpose robotics. We’re really focused on general-purpose methods. Some methods can be applied to both industrial or home robots or sidewalk robots, with all of those different embodiments and form factors. We’re not predicated on there being a general-purpose embodiment that does everything for you, more than if you have an embodiment that is very bespoke for your problem. It’s fine. We can quickly fine-tune it into solving the problem that you have, specifically. So this is a big question: Will general-purpose robots happen? That’s something a lot of people are tossing around hypotheses about, if and when it will happen.

Thus far there’s been more success with bespoke robots. I think, to some extent, the technology has not been there to enable more general-purpose robots to happen. Whether that’s where the business mode will take us is a very good question. I don’t think that question can be answered until we have more confidence in the technology behind it. That’s what we’re driving right now. We’re seeing more signs of life — that very general approaches that don’t depend on a specific embodiment are plausible. The latest thing we’ve done is this RTX project. We went around to a number of academic labs — I think we have 30 different partners now — and asked to look at their task and the data they’ve collected. Let’s pull that into a common repository of data, and let’s train a large model on top of it and see what happens.

What role will generative AI play in robotics?

I think it’s going to be very central. There was this large language model revolution. Everybody started asking whether we can use a lot of language models for robots, and I think it could have been very superficial. You know, “Let’s just pick up the fad of the day and figure out what we can do with it,” but it’s turned out to be extremely deep. The reason for that is, if you think about it, language models are not really about language. They’re about common sense reasoning and understanding of the everyday world. So, if a large language model knows you’re looking for a cup of coffee, you can probably find it in a cupboard in a kitchen or on a table.

Putting a coffee cup on a table makes sense. Putting a table on top of a coffee cup is nonsensical. It’s simple facts like that you don’t really think about, because they’re completely obvious to you. It’s always been really hard to communicate that to an embodied system. The knowledge is really, really hard to encode, while those large language models have that knowledge and encode it in a way that’s very accessible and we can use. So we’ve been able to take this common-sense reasoning and apply it to robot planning. We’ve been able to apply it to robot interactions, manipulations, human-robot interactions, and having an agent that has this common sense and can reason about things in a simulated environment, alongside with perception is really central to the robotics problem.

Simulation is probably a big part of collecting data for analysis.

Yeah. It’s one ingredient to this. The challenge with simulation is that then you need to bridge the simulation-to-reality gap. Simulations are an approximation of reality. It can be very difficult to make very precise and very reflective of reality. The physics of a simulator have to be good. The visual rendering of the reality in that simulation has to be very good. This is actually another area where generative AI is starting to make its mark. You can imagine instead of actually having to run a physics simulator, you just generate using image generation or a generative model of some kind.

Tye Brady recently told me Amazon is using simulation to generate packages.

That makes a lot of sense. And going forward, I think beyond just generating assets, you can imagine generating futures. Imagine what would happen if the robot did an action? And verifying that it’s actually doing the thing you wanted it to and using that as a way of planning for the future. It’s sort of like the robot dreaming, using generative models, as opposed to having to do it in the real world.