Santos Tour Down Under 2017 - 2017 National Finalist: Visit Sunshine Coast Best Tourism Event

Judging

processrulesjudgesback

The Australian Event Awards judging process is rigorous, transparent, and taken very seriously by the Independent Industry Judging Panel. The process is reviewed and refined each year to ensure its procedures continue to align with the Awards’ values of integrity, equity and inclusiveness, and maintain currency alongside the evolution of the industry.

Judging Process

This document outlines the Judging process which will be used for the 2024 Awards and should be read in conjunction with the Governance & General Rules. This document will be reviewed prior to the 2025 Awards.

Role of the Judging Panel

To see who is on the judging panel, click here.

  • Entries are judged and awards conferred by the members of the Independent Industry Judging Panel (IIJP).
  • The Managing Director of the Australian Event Awards Pty Ltd appoints and maintains the appointment of at least two Co-Chairs of the Independent Industry Judging Panel.
  • No member, director or employee of Australian Event Awards Pty Ltd or the Management Company may be a Co-Chair or member of the Independent Industry Judging Panel.
  • The Role of the Co-Chairs of the IIJP is to:
    • Determine the Categories, Eligibility Requirements and Judging Criteria of The Event Awards before the opening of entries each year
    • Determine the judging process to be applied to The Event Awards before the opening of entries each year
    • Address any questions regarding eligibility from entrants during the entry submission process
    • Appoint all other members of the Independent Industry Judging Panel
    • Inform and brief all judges on the judging process, criteria and expected outcomes
    • Rule on any real or perceived conflicts of interest in the judging panel
    • Lead the final judging meeting
    • Ensure the integrity and industry relevance of the Judging Process
    • Provide input and advice on the direction of The Event Awards generally
  • Appoint Industry Experts as Lead Judges to groups of categories as determined by the Co-Chairs
  • The Co-Chairs of the Judging Panel may, at any stage, appoint a person to act as a fact-checker to verify claims made in entries, and, following this, to advise the Judges and/or Lead Judges as to his or her findings.
  • Members of the IIJP may request of the Entrant (via the Management Company) additional information relating to the entry in order to accurately score and comment on the entry.
  • If there are particular issues and/or information in the public domain relating to an event or achievement which is featured in an Event Awards entry, this may be drawn to the attention of the IIJP.

Part 1: National Nominee Selection

Category Allocation

  • At the close of entries, each category will be allocated to one of the following groups (Category Group):
    • Public Events
    • Corporate / Business Events
    • Event Support Services
    • Design and Technical Services
    • Advice and Event Management
  • The Co-Chairs of the Judging Panel will appoint to each Category Group, a Lead Judge who is an expert in the field covered by the group.
  • Each Lead Judge is responsible for:
    • Allocating at least two Judges per category for review and scoring
    • Providing additional support to the Judges where required
    • Reviewing the Automatic Nominees list and undertake an in-depth review of the top two scoring Nominees in each category in their group, prior to Judges’ Review
    • Recommending to the Co-Chairs of the Judging Panel the winning entry in each Category.
  • Lead Judges may use industry knowledge of the event or achievement when scoring each entry.
  • Comments made in each Nominee entry by the Judges may be provided to the Lead Judge.

Scoring

  • Each entry is assigned to at least two Judges for scoring.
  • Each Judge checks the entries assigned to them and confirms that he/she does not have any real or perceived conflicts of interest in relation to the assigned entries.
  • If a Judge identifies any real or perceived conflicts of interest in relation to the assigned entries, the Judge must immediately notify the Management Company. The Management Company will then bring the conflict to the attention of the Co-Chairs of Judging who will evaluate the existence and seriousness of the conflict and rule that:
    1. no action is required OR
    2. the single entry should be reallocated to another Judge OR
    3. the conflicted Judge should be removed from judging any entries in that category
  • Judges are provided with a scoring guide such as the one below for the purposes of calibration:

 

Score Range (0-7) Guide
1-2 Demonstrates little or no understanding of the criteria or addresses it at an unsatisfactory level
3-4 Demonstrates satisfactory understanding of the criteria and addresses it at a satisfactory level
5-7 Demonstrates a good working understanding of the criteria and demonstrates a high level of achievement in addressing it
  • Each Judge assesses the entries assigned to them, scoring each entry against the criteria for its Category and providing comments to assist entrants with future entries.
  • As a Judge assesses further entries, they may wish to amend scores from previously-judged entries for consistency. This can be done at any time up until the close of the judging period.
  • Judges may use industry knowledge of the event or achievement when scoring each entry.

Nominee Threshold Score

  • The Nominee Threshold Score is set by the Co-Chairs of the IIJP to establish a quality standard that entries must achieve to be eligible to become a Nominee
  • To be eligible to become a Nominee (including a State or Territory Nominee), an entry must achieve a weighted average raw score that is at least equal to the Nominee Threshold Score.
  • The weighted average raw score is calculated by:
    • adding the raw score from each judge for each criterion
    • multiplying by the weighting of each criterion
    • adding the resulting weighted score for each criterion
    • then dividing by the number of judges for that entry

resulting in a single weighted average raw score for that entry. An example of this is illustrated in the table below:

Criteria Judge 1 Raw Score Judge 2 Raw Score
A 40% 3 4
B 20% 4 5
C 15% 6 5
D 15% 2 3
E 10% 4 3
Weighted Average Raw Score = 3.85
  • All entries (excluding National Events – those staged in four or more states or territories) accrue points towards the State or Territory Award regardless of whether they have met the Nominee Threshold Score.

Score Moderation

  • Scores for all entries are moderated using an academic scaling algorithm to remove any bias that may have been introduced by the use of different judges across the pool.
  • The result is a score out of 100 (Moderated Score) for each entry.

Automatic Nominee

  • The Entries that receive the top five Moderated Scores in each category are Automatic Nominee, providing they also meet the Nominee Threshold Score.

Nominee Review

  • Each Judge is advised of the Automatic Nominee in categories they judged and are invited to nominate any eligible entries that are not Automatic Nominee that they believe, against the published Category Criteria, are indistinguishable in quality from the Automatic Nominee and should be reconsidered for recognition as a Nominee.
  • Such a nomination must be supported by a statement from the nominating Judge that:
    • gives reasons for the nomination against the Category Criteria; and
    • puts forward the position that, in relation to the Category Criteria, the entry cannot be reasonably distinguished from the Automatic Nominees on the basis of quality.
  • To make such a nomination, the entry must:
    • Have been scored by the nominating Judge and received a score from that judge equal to or greater than the Nominee Threshold Score
    • Meet the Nominee Threshold Score
  • Entries nominated for review (Review Entries) are made available to a Review Panel made up of all judges that scored that Category (other than the nominating judge), and the Category Lead Judge. The nominating judge may be asked to participate as an advisor to the Review Panel.
  • In any Category where there is a Review Entry, Members of the Review Panel have the opportunity to re-read the Automatic Nominee entries together with any Review Entries. Members of the review panel may, in relation to each Review Entry:
    • support the nominating judge’s view that, in relation to the criteria, the entry is indistinguishable in quality from the Automatic Nominees and should be reconsidered for recognition as a Nominee
    • reject the nominating judge’s view that, in relation to the criteria, the entry is indistinguishable in quality from the Automatic Nominees and should be reconsidered for recognition as a Nominee
    • abstain from voting on the nomination (this is compulsory if the member of the Review Panel is conflicted in relation to any of the entries nominated for review or any of the Automatic Nominees for the Category)
  • If any Review Entry receives a majority vote of support, the Review Entry will become a Review Nominee for the Category.

Nominee Announcement

  • At the end of the Nominee Review, the Automatic Nominees together with any Review Nominees for each Category will become the Category Nominees for the Category.

Part 2: Winner Selection for all categories except the Headline Categories

Lead Judge Review

  • Prior to the Final Judging Meeting, each Lead Judge will individually undertake an in-depth review of the top two scoring Nominees in each Category in their Category Group.
  • The Lead Judge may contact a person nominated by the entrant if they feel further information or clarification is required.
  • The Lead Judge may also call for a Lead Judges Review where further discussion and clarification around the top two scoring entries is required. All Judges that scored that Category will be asked to participate.
  • The Lead Judge will decide on a recommendation for the Winner, based on the Category Criteria.
  • The recommendation, together with the scores of all Nominee entries in each category will be presented to the Co-Chairs of the IIJP ahead of the Final Judging Meeting.
  • Where the recommended winning entry is not the highest scoring entry, the Co-Chairs of the Judging Panel will read both the highest scoring entry and the recommended winner prior to the Final Judging Meeting.

Final Judging Meeting

  • Meetings will be convened for each Category Group.
  • Each meeting will be attended (either in person, or electronically) by the Co-Chairs of the IIJP and the Category Group Lead Judge.
  • Where a Lead Judge becomes unavailable for the Final Judging Meeting, the meeting will be rescheduled to an alternative suitable date in collaboration with the Management Company, Lead Judge and the Co-Chairs of the IIJP.
  • Where a Lead Judge becomes unavailable and rescheduling is not possible, the Management Company must assign any Categories for which the unavailable member was Coordinator to the Co-Chairs of the IIJP. The Co-Chairs then become the Lead Judges for the Category Group.
  • The final consideration of each Category proceeds as follows:
    • The Lead Judge introduces the recommended winner of the category and the next most highly regarded entry from amongst the Category Nominees.
    • The Co-Chairs and the Lead Judge each confirm that he/she does not have any real or perceived conflicts of interest in relation to the Category Nominees.
    • Where a member of the meeting has any real or perceived conflicts of interest in relation to any of the Category Nominees, the member must remove themselves from all discussion and voting regarding the Category.
    • The Lead Judge presents their assessment of the recommended winner of the category and the next most highly regarded entry from among the Category Nominees.
    • If requested by the Co-Chairs or the Lead Judge, a discussion may take place about the Category Nominees to cover any questions, concerns or submission of an alternative score to an entry in relation to the category criteria.
    • The Winning Entry is selected by vote from the Co-Chairs and Lead Judge.
    • The Winning Entry must receive a majority vote of support.
    • If no entry is able to receive a majority vote of support, the highest scoring nominee in the Category will be declared the winner.

Part 3: State and Territory Nominees and Winners

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process
  • All entries in the Best Event Award Categories are ranked according to Moderated Score.
  • Entries must meet the Nominee Threshold Score to become a State or Territory Nominee or Winner
  • National Events (those staged in four or more states or territories) are not eligible to become a State or Territory Nominee or Winner.
  • For each category, the two highest ranked events that occurred in each state or territory is awarded the State/Territory Nominee status
  • On the Awards night, after the National Nominee and Winners are announced in each category, the State and Territory Winners will be announced. Certificates for State and Territory Nominees and Winners will be available for collection during the Awards Ceremony, after the announcement of the category.

Part 4: Judges’ Special Award

The Judges’ Special Award is not open to nomination or entry from the general public or industry.

  1. Only Australian Event Awards Judges can make nominations for a Judges’ Special Award.
  2. A judge wishing to make a nomination will be required to provide to the Event Awards, 4 weeks prior to Final Judging Meeting, a brief overview of the nomination, for distribution to the Co-Chairs of the IIJP.
  3. Nominations accepted for review and consideration by the Co-Chairs will advised to the nominating Judges 1 week prior to Final Judging Meeting. Each nominating Judge will be required to attend a teleconference with the Co-Chairs of the IIJP at the Final Judging Meeting to discuss the nomination in detail.
  4. The Co-Chairs of the IIJP will consider each nomination and make the final decision on the awarding of a Judges’ Special Award.

Part 5: The State or Territory Award

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process.
  • National Events (those staged in four or more states or territories) are excluded from this category.
  • Every entry that is not a National Event in the Best Event categories contributes points towards the state(s) or territory(ies) in which the event was held.
  • To be considered, each State and Territory must meet the minimum score of 7 points and have at least two National Nominee events in their State or Territory, one of which must be a National Winner.
  • This award is awarded to the State or Territory which earns the largest number of points per head of population.
  • Each eligible entry submitted in the Best Event Award categories earns one point for the State or Territory in which their event was held. Each National Nominee entry earns an additional point (total: 2 points) and each National Winner earns an additional point (total: 3 points).
  • The State or Territory with the largest number of points per head of population will be awarded the ‘State or Territory Award’ provided that it has a minimum of 7 points and it hosted at least two National Nominees in the Best Event Award categories, one of which must be a National Winner.

Part 6: Event Supplier of the Year

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process.
  • The Winning Entries in the Industry Award Categories are eligible to be considered for this Award.
  • The Lead Judges responsible for the Industry Award categories will each provide a recommendation of a winner from their group of eligible categories to the Co-Chairs of the IIJP.
  • The Co-Chairs of the IIJP will review each recommendation and make the final decision on the winning entry.

Part 7: Event Management Company of the Year Winner

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process.
  • The Winning Entries in the Management Categories are eligible to be considered for this Award.
  • The Lead Judge for the Management Category Group will provide a recommendation of a winner from their group of categories to the Co-Chairs of the IIJP.
  • The Co-Chairs of the IIJP will review each recommendation and make the final decision on the winning entry.

Part 8: Australian Event of the Year Winner

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process.
  • The Winning Entries in the Best Event Award Categories are eligible to be considered for this Award.
  • The Lead Judges responsible for the Best Event Award categories will each provide a recommendation of a winner from their group of eligible categories to the Co-Chairs of the IIJP.
  • The Co-Chairs of the IIJP will review each recommendation and make the final decision on the winning entry.

Part 9: Unforeseen Circumstances

  • Should a ruling be required on a matter not covered in this document, the Co-Chairs of the IIJP will make a unanimous ruling on that matter
  • The Co-Chairs of the IIJP may take advice from judges or others as they see fit
  • Generally, resolutions to unforeseen circumstances are documented in the form of a revision to this document ahead of The Event Awards in the following year

Part 10: Appeals

  • The Event Awards does not have an appeals process. All decisions by the IIJP are final but the IIJP welcomes feedback and encourages constructive criticism aimed at improving The Event Awards.