Street view game - hit the road
1 The classic street view is too cumbersome
It was on a Sunday afternoon, as sunny as it gets on January afternoons. We were driving our car through a to us unknown countryside searching for a new home. The most frustrating about the entire experience was that we only had a vague idea of where to go and usually ended up on bigger roads. Sadly, that’s how well-designed street networks work. They should channel the traffic to bigger roads. This is also something I experienced in developing The road less travelled.
Back at home, I was thinking about how to make life easier.
Google maps to the rescue. Google made the gift of digitalizing much of the world with aerial maps, 3d maps and street view. Personally, I find the 3D maps really cool to get an overview of an area. Especially when planning your holidays, it can provide a better impression than a simple map with elevation levels.
But to explore neighbourhoods, I was much more interested in details and not an overview. That’s were Street view hits the nail. It is just Photos Sphere Photos, chained together. 2D VR does not get more realistic than a photo.
For me, Street view has a two cumbersome drawbacks. First, you need to click or press a key all the time. Second, the transition effect is visually fatiguing if you watch it several times. The transition effect is like a slide show fades. It is visually supporting, if you slowly transition from one point to the next. But if you hit the button to quick, the transition get’s distracting. In addition, for me rapidly pressing the forward key to let’s to a loading problem.
2 Searching for something better
There is one thing I particularly like about GitHub. Usually somebody else already solved your problem. You only need to get your head around his solution and then improve from there.
There is a custom C++ street view client. https://github.com/TheGreatRambler/streetview_client/
This application loads the full street view panoramas and then uses a custom stitching algorithm. This algorithm avoids the transition. By preloading an entire route, you can easily avoid any download lag. This seemed to solve my troubles with street view.
Then I had the idea. What if street view would be like a computer game? Why press the mouse button to turn and look around? In a game, you usually just move the mouse and use WASD to move around.
3 The first street view game: custom rendering
3.1 Rapid street view viewing
After spending a few evenings changing the code, I came up with my first street view based game. And it was true, the speed was amazing. You can see an animation here:
3.2 Usability and legality are important for a product
When turning an idea into a prototype, there is always the question of where to stop. How far should you continue and when should you share the solution with others? Personally, if I am sharing code, I try to maintain a minimal quality standard.
I decided against publicly sharing the code for this version because of some drawbacks.
3.2.1 Usability issues
The program requires the full download of the panorama, which can be 25MB. If you advance too quickly, the panorama hasn’t loaded yet. Google uses a very intelligent method to avoid this lag by only loading parts of the panorama that are visible. In the Google Street view, this effect is visible when quickly turning around.
Preloading everything in a radius solves this. A function that already exists in the GitHub version of the streeview_client. While it is working, it is not very practical. As already said, the high res street view panoramas are 25MB. In dense city areas, there are many panoramas, close to each other. If free exploration were to be possible, a lot of data needs to be loaded. Even loading all the panoramas along a predefined route takes a lot of time. The size of your temp directory quickly explodes. In addition, it works only with a quick internet connection.
Street view animation works by linking one panorama to the next. Sometimes a panorama hits a dead end, even if the street continues and, in a few hundred meters, another panorama exists.
Google maps solve this dead-end situation by requiring the user to just repositioning the peg man. In a custom engine, the only option would be to advance in the general direction and take the next panorama in that direction. Or reimplement the peg man map.
3.2.2 Legal issues
If you want to show street view data, you have to do it in a way that Google details in their documentation. And the custom render from the great rambler is probably not part of this.
4 Second Version: using the Google Maps API
4.1 Innovative use of the google api to reduce transition effect
As mentioned before, the main reason for not releasing the previous version publicly is that it risks violating the Google Maps’ API’s terms of service. Loading multiple panoramas on the same page seems to be allowed. As I detail below, you will just pay more for this.
The transition and loading problem is solved by using multiple panorama renderers. Basically you are looking at three panoramas. Depending on your internet connection, you can increase the or decrease the pause between the panoramas. If the loading pause becomes too small, the original transition effect becomes visible.
4.2 Travelling around
The current version solves the issues with the usability of street view by extending its functionality with the mouse turning, auto rotating, and auto forward. We always transition in the direction of view. If there are no further panoramas to the side, you can even look through the side window while advancing forward.
In google Street view you have to look at every panorama, which gets cumbersome on a highway. I use a jump distance to jump over some panoramas. Together with the reload frequency, this defines the speed of the “virtual vehicle”.
This leads me to one of the coolest use cases, besides exploring unknown areas. Traveling on scenic highways.
.
5 Technical feasibility is not enough
The hosted version of this project has a limited usage to remain within google’s free range for the google api and serve as demonstrator. It is just too costly to run it otherwise. Using multiple panoramas, you need to pay for every panorama you instantiate. Together with the lookup of the position, this puts the use at almost 3 cent per page load in 2024.
Advertisement generates revenue of 1c per page load for a well-working website. A mobile game rarely costs more than €5. All of these results in a loss for the app.
6 Hey Google, please help.
I still love the idea of using Google Street view like a video game. It would enable a lot more people to explore the world. I remember the time when Google Earth was new, and we just wanted it to explore unknown corners of the world. Exploring photo realistic content could be much more interesting. In fact, there are some applications for VR glasses. The application does not need VP glasses.
So if someone from Google Maps ever reads this: Please develop an official version.
Bonus points:
- Include 3D effects by relying on structure from motion
- Add a more intelligent routing, which can follow streets