How to Run a Judged Competition: Complete Organizer's Guide

Published July 14, 2026 · Updated July 14, 2026 · By the Continuous Cup team

A judged competition is any contest where a panel of judges scores entries against criteria: a coffee cupping, a homebrew or BBQ contest, a baking showdown, a science fair, a design award, a talent showcase. Running one well comes down to two jobs that pull against each other. You have to keep the logistics moving (entries, judges, schedules, scoresheets), and you have to protect fairness at every step so the winner deserves the trophy and everyone can see why.

This guide walks the full process in nine steps, from defining categories to publishing winners. It is the process itself; it applies whether you run it on paper, in a spreadsheet, or on purpose-built software. Where a step is much easier with software, we show how Continuous Cup handles it.

The nine steps at a glance

1. Define categories and eligibility
2. Collect entries
3. Choose scoring criteria and weights
4. Recruit and assign judges
5. Blind entries where appropriate
6. Normalize or aggregate scores
7. Handle ties and incomplete judging
8. Audit the results
9. Publish winners

Step 1: Define categories and eligibility

Decide what is actually being compared before anyone enters. A "best food" contest that mixes brisket against brownies produces winners nobody believes in. Split the field into categories (also called divisions or classes) where entries genuinely compete on the same terms, and write eligibility rules you can enforce: who may enter, how many entries per person, what disqualifies an entry, and what information each entry must include.

Keep the category list short enough that each category gets a real field. Three entries in a category is a coin flip with extra steps; if you expect thin categories, plan to merge them and say so in the rules up front.

How Continuous Cup handles it: competitions are set up with divisions and categories, per-entrant entry caps, and custom entry-form fields, so the rules you wrote are enforced by the entry form instead of by you at midnight.

Step 2: Collect entries

Give entrants one obvious place to enter, one deadline, and a form that captures everything you need the first time. Chasing missing details by email is where organizer hours disappear. If you charge an entry fee, collect it at submission so "entered but never paid" is not a category you manage by hand.

Physical entries (bottles, plates, bags of beans) need a second checkpoint: intake. Log each item as it arrives and tie it to its entry record immediately, because a table of unlabeled jars an hour before judging is how the wrong sample gets scored under the wrong code.

How Continuous Cup handles it: every competition gets a public entry page at a shareable link, with custom form fields, card payment for entry fees, and QR-code drop-off and check-in for physical entries. For county-fair style contests there is a single-day walk-up mode where entrants register and check in on-site. More on tasting-specific intake in the tasting competition guide.

Step 3: Choose scoring criteria and weights

A rubric turns "I liked it" into scores you can compare and defend. Pick four to eight criteria that describe what excellence means in this competition, give each a scale judges can use consistently, and weight the criteria by how much they matter. Weights are the honest part: a chili contest that is 40 percent flavor says something different from one that is 40 percent presentation.

Then write one or two sentences per criterion describing what a low, middle, and high score looks like, so two judges read the same scale the same way. Rubric design is its own craft; we cover it fully in how to create a judging rubric.

How Continuous Cup handles it: the rubric lives in the platform as weighted criteria. Every judge sees the same scoring form on their own device, and the weights are applied automatically when scores are combined, so there is no spreadsheet formula to get wrong.

Step 4: Recruit and assign judges

Recruit more judges than the minimum, because someone always cancels. Ask applicants about experience and conflicts of interest before you accept them: a judge who entered category 3, or whose student did, should not score category 3.

Assignment is the quietly technical part. Each entry should receive scores from several judges (three is a sensible floor), no judge should be overloaded past honest attention, and assignments should overlap: judges must share some entries in common, or there is no way to compare a strict judge with a generous one later. Doing this by hand for forty entries and nine judges is an evening of graph paper.

How Continuous Cup handles it: judge application forms with custom questions, email invitations, and automatic assignment that distributes entries across the panel with configurable overlap and rounds. Judges score from any device, and you watch coverage fill in live.

Step 5: Blind entries where appropriate

If judges know whose entry they are scoring, reputation scores the entry first. Blind judging replaces entrant identity with anonymous codes so the work is judged, not the name. It matters most exactly where it is hardest: small communities where everyone knows everyone's style.

Blinding is a chain, and it breaks at the weakest link: the intake sheet taped to the wall, the master key emailed around, the distinctive jar everyone recognizes. The fix is structural: identity and code should live in a system where judges simply never see the mapping, rather than in a document you hope nobody opens. The full treatment, including when open judging is legitimate, is in blind judging software.

How Continuous Cup handles it: entries are anonymized automatically and judges only ever see codes. For made-to-fit contests there is a fit-to-judge flow: judges share sizes or dietary needs, entrants make their item to match, and neither side learns who is who.

Step 6: Normalize or aggregate scores

Once scores are in, you have to combine them, and this is where fair competitions are won or lost. If every judge scored every entry, a plain average is fine. The moment different judges saw different entries, a raw average quietly punishes whoever drew the strict judge. That is not a rare edge case; it is every competition big enough to need multiple judging tables.

The standard tools, from simplest to strongest: trim the highest and lowest score and average the rest; normalize each judge's scores so strict and generous judges land on the same scale (robust z-score normalization is the dependable version); use rank-based methods that keep order but drop magnitude; or fit a Many-Facet Rasch model (MFRM), which estimates judge severity and entry quality together and is the same family of model used in high-stakes exam scoring. Plain-language explanations of all of these are in how to score a competition fairly.

How Continuous Cup handles it: organizers choose the method per competition: raw averages, robust z-score, rank-based, or MFRM, with optional trimming. Scores tally automatically as judging finishes, and the same math runs every time.

Step 7: Handle ties and incomplete judging

Two things will happen at the worst moment: two entries will tie, and a judge will go home with scoresheets unfinished. Both are rules questions, and the only wrong answer is inventing the rule after you know who it helps. Decide before the event how ties break (a priority criterion, head-to-head comparison, a taste-off round, or shared placement) and put it in the published rules.

For incomplete judging, decide the minimum coverage an entry needs to place, and what happens below it: another judge picks up the entry, or the entry's result is computed from the judges it did get, using a method that tolerates unequal panels. This is another reason overlap in step 4 matters: methods like MFRM handle uneven coverage gracefully, but only if assignments overlap enough to compare judges.

How Continuous Cup handles it: tiebreakers are configured up front, coverage is visible while judging runs so gaps surface in time to fix, and the scoring methods are built for panels where not everyone scored everything.

Step 8: Audit the results

Before anything goes public, look at the results the way a skeptical entrant will. Did any judge score far from the panel on the same entries? Did one table run hot? Does the winner hold up if you recompute with the highest and lowest scores trimmed? Ten minutes of this before publishing prevents the week of email after.

How Continuous Cup handles it: every judge is automatically compared with the rest of the panel using robust statistics that a single unusual score cannot throw off, and judges who consistently drift get a judge-bias flag on the results report before you publish. A defensibility report captures how the result was computed, so the answer to "prove it" already exists.

Step 9: Publish winners

Publish results somewhere permanent that anyone can check, not a screenshot in a social post. State the category, the placement, and, for competitions that want to be trusted, how scores were combined. Then close the loop with entrants: winners get certificates or credentials they can share, and every entrant should be able to find their own result without emailing you.

How Continuous Cup handles it: results publish to a public page with a step-by-step judging-transparency view, winners get certificates and verifiable credentials, and entrants are notified by email. The public page is also your archive: next year's entrants can see how last year was scored.

The organizer's checklist

• Categories defined, eligibility and entry limits written down, thin-category merge rule published
• One entry link, one deadline, fees collected at submission
• Rubric: 4 to 8 weighted criteria with level descriptors, piloted before the event
• Judges recruited with conflicts declared; every entry covered by 3+ judges; assignments overlap
• Entries blinded with codes; the identity mapping lives where judges cannot see it
• Scoring method chosen and announced before judging starts
• Tiebreak and incomplete-judging rules in the published rules
• Bias check and recompute-with-trim sanity pass before publishing
• Results on a permanent public page; certificates and notifications out

If you would rather run this checklist than build it, that is the product. Start with the competition judging software overview, see how it compares with running your competition in a spreadsheet, or set up a competition free and walk through the guided setup.