512kg / dima
Commercial

Ship small tools

A downloader for Pinterest boards that keeps the original image quality. It exists as a website and as a Chrome extension, with one subscription for both. Most of what makes it better than the alternatives came from measuring instead of assuming.

Why other tools hand you thumbnails

Almost all of them read the thumbnail from the page and rewrite its address, swapping the size for the folder of originals. The thumbnail is always a .jpg. The original is often a .png or a .gif, so the rewritten address does not exist and the server answers 403. When that happens they keep the thumbnail and say nothing.

Measured on real boards: on a board of photographs, 30 of 52 rewritten addresses failed. On a board of graphics, 18 of 48 failed and 10 more GIFs came back as a frozen JPEG.

This tool asks for the real address of each file, with its correct extension.

One pass is not enough

This was the finding that changed the result most. Reading a board once does not return the board. Each attempt returns a different subset, with a lot of overlap. On a test board of 580 pins:

Fig. 1 — Images returned from a 580-pin board, by approach. Measured on one test board. The lighter part of the second bar is the spread between attempts. 446 is the most an anonymous visitor reached, joining many passes; only a signed-in session returned all 580.

Two causes were mine and are fixed: fixing the page size made the distribution worse, and reading once left out a third of the board. The third is not mine. Around 446 is what an anonymous visitor can see, and no amount of retrying gets past it. With a signed-in session, which only the extension has, the same board returns 580 of 580.

The lesson I took from it: a short result looked exactly like a correct one. There was no error anywhere. So the tool now compares what it collected with what the board declares, and if it has less than half it says so, instead of presenting three images as if that were everything.

One subscription, nothing to type

The subscription is valid on the site and in the extension, and the licence travels by itself:

  1. The customer pays. A Stripe webhook creates the licence, and the site stores its key in a cookie.
  2. The cookie is HttpOnly, so not even the page’s own scripts can read it. A small bridge script asks the server for the key, and the server only answers a request from its own origin with a live licence.
  3. The bridge hands the key to the extension over a channel that is pinned to the identifier of the official extension.
  4. The extension validates the key against the server before saving it, and revalidates every twelve hours. A cancellation stops working the same day.

An honest limit, not a lock

The extension’s free allowance, five a day, is counted on the user’s own machine. That is deliberate. Counting it on the server would mean telling the server about every download, and the promise of the extension is precisely that it does not. Without a subscription the extension does not contact the site at all.

Anyone who wants to get round that limit can. I decided I would rather have a limit that respects the user and can be bypassed than one that cannot be bypassed and watches them. On the website, where requests reach my server anyway, the counter is exact. How that one is built.

What the first real use uncovered

  • 580 confirmation dialogs. With “ask where to save each file” turned on in the browser, every download opened a dialog. One explicit option in the download call ignores that preference.
  • A minute of waiting for nothing. After finding all 580, it read the board twice more to confirm nothing new appeared. Now it stops as soon as it reaches the declared total.
  • The whole extension rejected, with no reason given. The manifest declared a default language without the folder of translations. The browser refuses the entire extension in that case and does not say why.

The parts that have to be exact

  • The image proxy only accepts the addresses of original files.
  • Thumbnails go through a route that decides the size on the server, so an original cannot be pulled out through it.
  • The lookup endpoints have a daily cap per visitor. Without it they were an open scraper running on my quota. A large board uses about 200 calls and the cap is 1,000, so real use never notices.
  • Pro costs $5 a month. The free plan on the site is five originals a day, and browsing a board costs nothing.
  • HTML63.2%
  • JavaScript36.8%