How we make games

Each game starts as a written brief: its rules, its scoring and how it gets harder. We aim for a first run anyone can finish.

The rules are kept apart from the drawing, in a small program that only ever reads the round's seed and your inputs. It has no access to the clock or to the browser's random numbers, so the same start and the same inputs always give the same result, on any device. That is what makes the replay check further down possible.

Every game then goes through an automated check. Its rules have tests of their own. Saved runs have to replay to exactly the same end. A test player has to score better than tapping at random and better than mashing, and its runs over many rounds set what counts as a good score. A browser loads the built game, starts it and plays it for a few seconds, and the check fails on an error or a blank screen.

A game labelled for phones has to work well on one: we keep a written standard for it, with minimum sizes for anything you touch. A game that plays on a computer gets a layout for a big screen.

People on the team play every game on real phones and computers, and every game has a place on one shared board with its ratings and notes. A lot of our reworks start from those notes.

A daily game gives every player the same round for one day, and a new day starts at midnight New York time. Each game draws its days from its own pool of questions, puzzles or places, by rules written for that game.

A question should not come back within a year. While a pool is still growing we hold to a lower floor: 120 days, or 60 days for answer games whose days are drawn at random from a big pool. Those answer games also start a day with easy prompts and mix subjects and kinds of question, and Uncommet limits how much of a day can rest on one person's works or on US knowledge.

Puzzles the game generates are checked before you see them. Sumdinger's sums are checked by a solver that finds an exact answer, Daily IQ keeps a puzzle only when exactly one option fits, and every box in Pack It has been packed clean by a test player using the same physics you play with.

You get one go at each day's round, on each account and each browser, so a score on the day's board is a first try.

The Daily is separate: three games, the same levels for every player and one attempt at the set. It deals levels of its own, never the round a game's own day is showing, and its seeds are made with a secret, so its levels can't be worked out in advance. No game is dealt into The Daily two days in a row.

When you play a game with a leaderboard, the game records what you pressed and when, not just the score. Our servers then run the same version of the game, the exact code you played, identified by its hash, with those inputs and nothing else. Clocks and random numbers are switched off there, so the rerun cannot drift. The score, the length of the run and the checkpoints along the way all have to match.

Every scored run that can change a board gets this check, guests' runs included. A run still waiting for it is marked unverified. A run that fails comes off the board and is flagged, and so does a run that replays identically to someone else's. On The Daily the game also sends a checksum of your inputs about every ten seconds as you play, so a run cannot be rewritten afterward or played faster than real time.

A replay proves that a score came from those inputs. It cannot prove who or what pressed them. So bots and scripts are against our terms of use, and on our longer games the top 10 of each board also goes into a review queue, where a person can hide a run. In question games the clock keeps running while a question is open, so it can't be stopped to look an answer up.

Our questions and lists are built from open sources we are allowed to use: Wikidata, public data from agencies such as the US Census and NASA, Natural Earth and GeoNames for places, the xkcd color survey for color names and the Lichess database for chess positions. We also read Wikipedia for lists and to check facts, and never copy its text. Each game's data records its sources, licenses and dates.

A fact needs a second, independent source before it ships. The one exception is a plain statement about an everyday thing, which may rest on the opening of its Wikipedia article. That statement is then confirmed separately against the article as it reads that day, and any number, date or record in it still needs a second source.

New batches go through a blind check that sees the question but not its source. It covers at least one item in ten of every new batch and all of a game's first batch, and when something fails we fix it and sample the batch again. Our written trivia questions get this on every item, against two independent sites, and every brain teaser is solved a second time without seeing the answer before the two are compared.

An answer counts only if it really belongs to the category as the question words it: a town is not a city unless the question says "city or town". Lists aim for a category's real size, so a right answer isn't refused, and when one is, it goes on the list in a later fix. Time-sensitive facts carry an as-of date. If you find something wrong, tell us through the feedback form.