How we built the Waifu Vending Machine
On the weekend of Anime Expo, an artist promised the humanly impossible —- custom art commissions with a five-minute turnaround time. The not-so-hidden catch? The artist wasn’t a human at all, but an AI exposed to millions of anime images. Over the next four days, what started as a simple experiment would go on to become a cult attraction in the artist alley, drawing lines of up to 40 minutes and forcing us into a race against time to scale up our system.
In this post, we’ll be going over how the Waifu Vending Machine was built – from the initial design process, to the behind-the-scenes of booth construction, to the last-minute hacks that saved the day. (Special thanks to our neighbors and the wonderful staff at artist alley for putting up with our traffic)
Note: this is not a technical report. Instead of focusing on ML techniques, we’re going to focus on the less-explored question of taking an isolated model and bringing it to life!
What is the Vending Machine anyways?
In anime convention culture, the artist commissions are always a center of attraction. For a price, artists will draw the whatever character you describe – from current trends to the rarest girls from the 90’s. When exchanging a commission, however, it’s frequently a challenge to describe the character you’re imagining, and it can often take days before the finished product is ready.
The Waifu Vending Machine is our re-imagination of the traditional commission booth. Rather than dealing with communication troubles and overnight delays, the process of the Vending Machine is simple – select the girls you like, and the booth will learn from your choices to create the character of your dreams!
Walk up to the station, and you’ll be greeted with a quick array of girls.
The search for perfection
After each step, the booth narrows your choices – eventually leading to a final screen, where you can “adopt” the girl on the spot.
And she’s ready!
At first, the Vending Machine was just a small experiment to see if we could shake up the artist alley. From hastily-drawn napkin plans, we built everything up, hoping to bring smiles to a few passerbys. However, word-of-mouth soon transformed our booth into a prime attraction in the alley – and our team had to pull out all the tricks in the book and more to keep the booth afloat.
Crowds arrive!!!
For the rest of this post, we’ll describe the behind-the-scenes adventure of the Waifu Vending Machine creation. It was a journey of unexpected amazements!
The Prototype
The Vending Machine project started out as a simple idea on the research checklist – figure out how to generate an infinite source of anime girls, from scratch.
The initial results were pretty, but the only thing we could really do was look at the results and say “yeah that looks cool”.
We didn’t want to generate just any girls – we wanted to draw everyone’s dream girl. For that, we had build knobs to guide the drawing process. To make things a little more interactive, we spun up a quick tool to mix and match parts of the generated girls. By exchanging parts of the girl’s representations, we could do things like change color palette and hair style.
Beautiful UI
This was fun, but finding the girl we were thinking about could still take ages. What we needed was an easier way to specify what kind of girl we were looking for…
Plans for success
The problem with building some kind of fine-grained control dashboard is that people are bad at describing what they’re thinking about. On the other hand, when given clear options, it’s much easier to choose which option is the best.
Following this logic, we opted for a system where the user is shown a grid of girls, and the system would narrow down their search with every selection.
The final UI! Or is it…
With that, we had a UI good enough to begin playing around with! It’s a simple workflow where people figure out what they want by selecting girls incrementally closer to their dream girl.
With software out of the way (or so we thought), it was time for the hard stuff.
The Hard(ware) Stuff
Even before arriving at the artist alley, there were some challenges we had to address. The first was how to get noticed in a sea of artists and posters. To really stand out, we’d need something that could both catch attention and convey the nature of the booth.
Precise design specs
We ended up drawing inspiration from a common resident of the city buildings: the vending machine! With its towering height and theme of automation, we hoped the vending-machine design could convey our ideas with a single glance.
Of course, we wouldn’t be able to build an actual machine – we would have to hack it. This is a theme that became increasingly common as the days drew closer.
The final design was essentially a guy sitting behind a box-shaped cloth backdrop, while the monitor in the front ran a UI that people could interact with. Behind the scenes, our cluster of laptops and cables would power the core operations.
Our mascot in full form
The Battle
Finally, the promised weekend had arrived. After a six-hour drive down to LA, booth materials packed in the trunk, we walked into the convention center ready to begin the fight!
LineCon welcomes us with open arms
As it turns out, artist alley is an environment of incredible scarcity. Two factors stood in the way of the booth’s opening – there’s no electricity, and there’s no wifi. We also had around 6 feet of space, which was a pretty cramped space to sit in for 10 hours.
Thankfully, our booth was compact enough to fit on the table snugly.
Three key areas faced the user: the monitor with the ordering UI, a slot for money, and a place to collect the product.
Behind the scenes, the entire setup for the vending machine was a cluster of six laptops, each with enough battery life to last a couple of hours. By switching between them, we could survive the day.
On the right, a client laptop controlled what the users were seeing. It mirrored the display in the front so we could keep track of what the user was doing at. any time. On the left, the server laptop displayed the database logs and connected to the nearby photo printer, which created the portraits we then cut out.
The actual server, unfortunately, wasn’t some magical machine in the cloud. Since the internet wasn’t going to cooperate, we would have to source everything locally. The trusty gaming laptop, complete with its mobile GPU and copy of Windows Home Edition, was repurposed as the world’s most unstable inference server.
The Server
In the end, a typical loop would go something like:
- User walks up to the outward monitor and activates the machine
- We watch the mirrored display from the back
- When the user clicks submit, we read their image from the database and start a print job
- While the print job is underway, we handle the payment through a Stripe terminal and stall for some time
- Finally, we cut-out the print, sign the girl’s name on the certificate, and send the results through the flap!
From behind the booth, every machine is in arm’s reach.
By the end of the expo, we had this routine down to an art. Especially when multiple users started using the system at once, it became increasingly important to cut out any wasted time in the core loop – and even then, there was always the chance that something could break and send us all into a panic….
Top 10 Hacks
What follows next are series of techniques we performed out of sheer necessity.
Two hours into setup, we realized that the internet problem was worse than we thought. Not only could we not access the outside world, we couldn’t even use the building’s wifi as a local network. So in a last minute effort to connect everything together, we hosted a hotspot on an Android phone, disconnected the phone from data, and used it as a local router.
As usual, the Windows machine complained, and began to disconnect itself from the network. To stop this, we launched a perpetual ping job to the Windows IP, to keep it from refreshing its connection.
Cluster control
An hour into the first day, wed realize there was a critical bug in the front-end code – if people clicked the Back button on the first screen, the client would crash. Hastily, we redirected the client laptop to load the webpage from a different computer, and we hot-fixed the code from behind the scenes.
Two laptops, twice the efficiency
The final great hack occurred when we had to transfer a new video from one laptop to another to update the splash screen. We attempted an Airdrop, but the connections refused to show up. Instead, we hosted a python -m SimpleHTTPServer on one machine, then downloaded the file from the other machine directly from the local filesystem.
Iterate! Iterate! Iterate!
On the first day, we found it hard to attract people to the booth. The initial screen was confusing, and it became clear that people didn’t understand what was going on.
Home Screen
Improving the flow was tricky. We wanted to edit the UI, but we couldn’t risk breaking the experience for the users who were playing with the system out in front. Instead, we made use of the hot-fix system from before. While two of us manned the machine, the other two were hard at work in the back, pushing updates in real time and polishing up the experience.
Home Screen (v2)
For anyone who visited the first day, all we had was a simple 16-girl grid. By the afternoon, we had implemented an Undo button, and by the dawn of Friday, users could restart the process, refresh the grid, and even restore their girl from a backup if the system crashed.
A Heroic Age
After an absolutely hectic first day of running around and patching issues, we entered Friday with little expectations. But the word had spread – and apparently, the design of booth paid off greatly. People were intrigued by the idea of a vending machine as much as they were by the prospect of AI-powered designer commissions. As friends brought more friends, we soon had a perpetual crowd surrounding the front of the table.
To handle this influx of people, we converted two of the laptops to clients, laying them out in the tables next to the machine. Of course, this meant that server was under even more load – at some point, we even used the printer (??) as a router when the Android device began giving away.
But at this point, these were all a good problems to have. As a ship, we were constantly patching holes in the hull to prevent a sink. But at the same time, we were moving at speeds that were far beyond expectation, and the breeze of wind quickly blew all our worries away.
At the end of the expo, the Waifu Vending Machine sold out of all its stock, twice. It was a weekend of amazement, and despite the challenges, it’s great to be able to say we pushed through and let the commission booth of the future shine.
The Future and Beyond Time
During the expo, we promised to everyone that we would have the Vending Machine up online soon. After a bunch of reworks, the site is now up – check us out at waifulabs.com!
Thanks to everyone who came to check us out, and we’ll see you again at the next stage.
