The Gift Machine

Motivation

During Christmas 2022 I felt overwhelmed from having to come up with presents ideas in a rush. Due to my usual lack of time I had not thought about presents until the very last days. As always I ended up buying whatever I came across within one frantic run through the mega mall, spending a lot for presents nobody really liked very much.

The Idea

I then had the inspiration to create a website that comes up with present ideas for each member of the party, with the possibility to later revise suggestions, monitor the overall spending budget and eventually add the items to the amazon cart for purchase. Coincidentally, the idea came when ChatGPT was launched, so this was a perfect occasion to develop something with it, as the idea generation task appeared much more approachable with the help of AI.

Acquiring Gift Ideas and Product data

I immediately tested out some prompts and ended up generating 50 product ideas with the OpenAI API, creating one batch for five different age bins, five different budget classes and differentiating for sex, ending up with a total of 2500 (i.e. 50 * 5 * 5 * 2) ideas. I used a huge JSON file as a permanent storage to archive the ideas.

Next I wrote a parser to go on amazon with the gift idea and extract the first page of products, saving the price, ratings and number of reviews for each one in the JSON database. The parser was the most difficult part, as Amazon does not like to be accessed programmatically and any method would not work for long. I ended up developing and testing these methods to query the site:

• python request module with user agent request parameters
• parallel requests piped through TOR
• playwright javascript module

Of all the paths I have tried, playwright showed the most success. However, in each of these methods there was a constant process of trial and error, manually timing the requests and dividing them into batches, waiting out the temporary bans due to exceeding rate limits.

Since I was developing the parser along the way, I persisted the pages source into a PostgreSQL database so that I could try to improve the parser along the way, while avoiding to launch the site scraping every time.

The parser was a bit of brute force approach: using the classic beautifulsoup library the list is divided into individual products, then an array of micro-parsers is run against each product to extract individual bits of information. The micro-parsers allowed me to use different techniques to extract the data from the HTML: where the first micro-parser fails, another one is run to attempt extract the same piece of information, and so on. This improves the overall probability to obtain product data.

The below image represents the progress of the parsing process: each square corresponds to an idea, so that the 2500 ideas are organized into a 50x50 matrix. The color of each square represents the number of successfully parsed products.

As the image was taken while developing the parser, about one third of the matrix to the left is darker, meaning that the parser was still not able to decode everything and gave back fewer products with all parameters populated.

Developing the API

With the product database completed, it was time to put together a small API to serve the products. This was incredibly easy thanks to the excellent fastapi package. Everything in the API server worked out of the box and I had only to provide the function definition to return a random idea and product from a JSON dump of the database, based upon budget, age and sex. Brilliant!

Developing the Front End

This project has been an opportunity for me to learn about front end development, and for this I chose to try Svelte, as it was the most developer friendly option available.

During development, the framework got a major change with the introduction of “runes”, so I refactored the code and embraced the new approach. Svelte adoption rate has stalled over the course of 2024, perhaps because the production version of the runes redesign was long in the works and to some extent has made the framework similar to its bigger competitor React.

Regardless of how Svelte and React will end up in 2025, it was fun developing for the browser. For the UI\UX I designed as I went on (yes, bad design from an Engineer), but with the foundations of Tailwind CSS and DaisyUI I managed not to completely screw it up!

After the usual landing page, the user is simply prompted to provide a list of people with budget, sex and age groups, and with this information the app generates the gift list. There is one API call per person in the list, which is repeated in case the user wants to shuffle a particular gift to see other ideas and products.

People and Gifts are saved in the browser local storage, which is used instead of the login mechanism to save a sort of session for the user. Responsive UI was then implemented with the help of AI.

Deployment and Overall Stack

Deployment was probably the hardest part for me. There are so many contending and overlapping solutions on the market, each of them requiring at least a bit of a learning curve to be productive.

Having no server side processing in svelte, cloudflare pages was ideal to host the front end. I figured out that I could deploy the server on Google Cloud Platform, which requires building a docker image locally and then deploying it in the cloud.

This process is far from being user friendly and linear, requiring the developer to figure out many steps through forums and tons of documentation of several tools. The most confusing bit was having to develop two distinct docker images, one for my mac to test locally and another for the remote server architecture target.

Also, uploading changes to the database or to the API is still rather cumbersome: a new docker image has to be built each time, requiring to create a 1.5GB file each time, upload to cloud, find the instance in Artifact Registry and set that image in the GCP runtime. I really wish there was an easier method for this!

Here below is the overall tech stack for the project components.

The result

After figuring out the development and the deployment of the app, it is live here: The Gift Machine. See below a little demo preview of what the app can do.

How is it going?

Seeing this project more broadly, it was not only the occasion to pick up web development once again, but also to take action into making an idea reality.

I’ll start with the hard truth: during the first month that the website has been live, there have been barely any views, and consequently no purchase has been made through it.

The ‘marketing’ efforts were simply some basic SEO optimization which did not result in organic traffic, and a couple posts on reddit.com. Certainly I could have done better on this front.

There are several lessons that I have learnt from this experience, that I would like to share in the following section.

Lessons

What initially seemed a cool idea and an easy side project was hiding some issues which could have been intercepted from the beginning. From an entrepreneurial point of view I missed that:

• the use case is highly seasonal, probably peaking during winter;
• the website monetization through affiliation is highly dependent on amazon.com;
• the capability of generating site content is also highly dependent on amazon.com;
• research is needed to verify there is demand before developing;
• having a viable marketing strategy is as important as the app itself.

Now What?

I don’t consider the project ‘archived’ yet, as there is a whole new year ahead to improve it, market it and have it ready for next Christmas season, where it is more likely to be useful to people and searched for.

I have so far gained a couple user feedbacks regarding the following:

• the “newborn”, “baby”, “teen”, etc. age bins are missing from the app
• the categories are too generic and need to be personalized with interests

While the first feedback is reasonable and much achievable with what is already implemented, the latter is much harder since the products have to be precomputed before the real time user interaction.

Amazon have a dedicated API (Product Advertising API) for searching programmatically (i.e. instead of scraping), but it seems it is gated behind the fact that you must already have at least 3 qualifying sales through the site, generating a chicken-and-egg situation.

I hope you learnt something by reading about my experience, feel free to leave any advice or feedback in the comment section below, and of course tell me if you tested the app!