Venues Canberra for their work at Exhibition Park in Canberra - 2017 National Finalist: Best Achievement in Venue Management

Judging

processrulesjudgesback

The Australian Event Awards judging process is rigorous, transparent, and taken very seriously by the Independent Industry Judging Panel. The process is reviewed and refined each year to ensure its procedures continue to align with the Awards’ values of integrity, equity and inclusiveness, and maintain currency alongside the evolution of the industry.

Eligibility

  • Entries into The Event Awards for 2018 are open to events staged in Australia and achievements by Australians during the period from 1 July 2017 to 30 June 2018.
  • The event or events that is the subject of the entry must have taken place wholly between 1 July 2017 and 30 June 2018 or, in the case that part of the event falls inside and part outside of these dates, the entrant must contact The Event Awards for a ruling from the Judging Panel on whether they are eligible to enter in a particular year.

Judging Process

This document outlines the Judging process which will be used for the 2018 Awards and should be read in conjunction with the Governance & General Rules. This document will be reviewed prior to the 2019 Awards.

Role of the Judging Panel

To see who is on the judging panel, click here.

  • Entries are judged and awards conferred by the members of the Independent Industry Judging Panel (IIJP).
  • The Managing Director of the Australian Event Awards Pty Ltd appoints and maintains the appointment of at least two Co-Chairs of the Independent Industry Judging Panel.
  • No member, director or employee of Australian Event Awards Pty Ltd or the Management Company may be a Co-Chair or member of the Independent Industry Judging Panel.
  • The Role of the Co-Chairs of the IIJP is to:
    • Determine the Categories, Eligibility Requirements and Judging Criteria of The Event Awards before the opening of entries each year
    • Determine the judging process to be applied to The Event Awards before the opening of entries each year
    • Address any questions regarding eligibility from entrants during the entry submission process
    • Appoint all other members of the Independent Industry Judging Panel
    • Inform and brief all judges on the judging process, criteria and expected outcomes
    • Rule on any real or perceived conflicts of interest in the judging panel
    • Lead the final judging day
    • Ensure the integrity and industry relevance of the Judging Process
    • Provide input and advice on the direction of The Event Awards generally
  • The IIJP is divided into two sub-panels:
    • a Finalist Selection Panel (FSP) and
    • a Winner Selection Panel (WSP)
  • Members of the IIJP can be appointed as a member of either sub-panel or both sub-panels by the Co-Chairs of the IIJP.
  • The Co-Chairs of the Judging Panel may, at any stage, appoint a person to act as a fact-checker to verify claims made in entries, and, following this, to advise the judges on either the FSP or the WSP as to his or her findings.
  • If there are particular issues in the public domain relating to an event or achievement which is featured in an Event Awards entry, this may be drawn to the attention of the IIJP.

Part 1: Finalist Selection

Scoring

  • Each entry is assigned to at least two members of the FSP (FSP Judges) for scoring.
  • Each FSP Judge checks the entries assigned to them and confirms that he/she does not have any real or perceived conflicts of interest in relation to the assigned entries.
  • If a member of the FSP identifies any real or perceived conflicts of interest in relation to the assigned entries, the Judge must immediately notify the Management Company. The Management Company will then bring the conflict to the attention of the Co-Chairs of Judging who will evaluate the existence and seriousness of the conflict and rule that:
    1. no action is required OR
    2. the single entry should be reallocated to another FSP judge OR
    3. the conflicted FSP judge should be removed from judging any entries in that category
  • Members of the FSP are provided with a scoring guide such as the one below for the purposes of calibration:

 

Score Range (0-7) Guide
1-2 Demonstrates little or no understanding of the criteria or addresses it at an unsatisfactory level
3-4 Demonstrates satisfactory understanding of the criteria and addresses it at a satisfactory level
5-7 Demonstrates a good working understanding of the criteria and demonstrates a high level of achievement in addressing it
  • Each FSP Judge assesses the entries assigned to them, scoring each entry against the criteria for its Category and providing comments to assist entrants with future entries.
  • As an FSP Judge assesses further entries, they may wish to amend scores from previously-judged entries for consistency. This can be done at any time up until the close of the judging period.
  • Judges may use industry knowledge of the event or achievement when scoring each entry.

Finalist Threshold Score

  • The Finalist Threshold Score is set by the Co-Chairs of the IIJP to establish a quality standard that entries must achieve to be eligible to become a Finalist
  • To be eligible to become a Finalist (including a State or Territory Finalist), an entry must achieve a weighted average raw score that is at least equal to the Finalist Threshold Score.
  • The weighted average raw score is calculated by:
    • adding the raw score from each judge for each criterion
    • multiplying by the weighting of each criterion
    • adding the resulting weighted score for each criterion
    • then dividing by the number of judges for that entry

resulting in a single weighted average raw score for that entry. An example of this is illustrated in the table below:

Criteria Judge 1 Raw Score Judge 2 Raw Score
A 40% 3 4
B 20% 4 5
C 15% 6 5
D 15% 2 3
E 10% 4 3
Weighted Average Raw Score = 3.85
  • All entries (excluding National Events – those staged in four or more states or territories) accrue points towards the State or Territory Award regardless of whether they have met the Finalist Threshold Score.

Score Moderation

  • Scores for all entries are moderated using an academic scaling algorithm to remove any bias that may have been introduced by the use of different judges across the pool.
  • The result is a score out of 100 (Moderated Score) for each entry.

Automatic Finalists

  • The Entries that receive the top four Moderated Scores in each category are Automatic Finalists, providing they also meet the Finalist Threshold Score.

Finalist Review

  • Each member of the FSP is advised of the Automatic Finalists and members are invited to nominate any entries that are not Automatic Finalists that they believe, against the published Category Criteria, should be considered by the Winner Selection Panel (WSP).
  • Such a nomination must be supported by a statement from the nominating FSP Judge that:
    • gives reasons for the nomination against the Category Criteria; and
    • puts forward the position that, in relation to the Category Criteria, the entry cannot be reasonably distinguished from the Automatic Finalists on the basis of quality.
  • To make such a nomination, the entry must:
    • Have been scored by the nominating judge and received a score from that judge equal to or greater than the Finalist Threshold Score
    • Meet the Finalist Threshold Score
  • Entries nominated for review (Review Entries) are made available to a Review Panel made up of all judges that scored that Category (other than the nominating judge), and the Co-Chairs of the IIJP. The nominating judge may be asked to participate as an advisor to the Review Panel.
  • In any Category where there is a Review Entry, Members of the Review Panel have the opportunity to re-read the Automatic Finalist entries together with any Review Entries. Members of the review panel may, in relation to each Review Entry:
    • support the nominating judge’s view that, in relation to the criteria, the entry should be considered by the WSP
    • reject the nominating judge’s view that, in relation to the criteria, the entry should be considered by the WSP
    • abstain from voting on the nomination (this is compulsory if the member of the Review Panel is conflicted in relation to any of the entries nominated for review or any of the Automatic Finalists for the Category)
  • If any Review Entry receives:
    • at least one vote of support or nominations from two judges; and
      • no votes of rejection

the Review Entry will become a Review Finalist for the Category.

Finalist Announcement

  • At the end of the Finalist Review, the Automatic Finalists together with any Review Finalists for each Category will become the Category Finalists for the Category and the list of Category Finalists will be released.
  • From this point, no distinction will be drawn between Automatic Finalists and Review Finalists.

Part 2: Winner Selection for all categories except State or Territory Award and Australia’s Favourite Event

Category Allocation

  • Each member of the Winner Selection Panel (WSP) is allocated a number of Categories to review and coordinate.
  • Each member of the WSP checks the Category Finalists in the allocated Categories and confirms that he/she does not have any real or perceived conflicts of interest in relation to the entries.
  • If a member of the WSP identifies any real or perceived conflicts of interest in relation to any Category Finalists in an allocated Category, the Judge must immediately notify the Management Company. The Management Company will then bring the conflict to the attention of the Co-Chairs of Judging who will evaluate the existence and seriousness of the conflict and rule that:
    1. no action is required OR
    2. the conflicted judge should be removed from judging any entries in that category
  • Once the confirmation above is made, the member of the WSP becomes the Category Coordinator for each category they’ve been assigned.
  • This means they are responsible for undertaking detailed review of the Finalist entries in each of their categories prior to Final Judging Day. They will represent the categories assigned to them and, for each category, lead the process described below.
  • Members of the WSP may use industry knowledge of the event or achievement when scoring each entry.
  • Comments made in each Finalist entry by the FSP Judges may be provided to the WSP Category Coordinator.

Final Judging Day

  • All members of the WSP attend the final judging day.
  • Where a member of the WSP becomes unavailable for the final judging day, the Management Company must assign any Categories for which the unavailable member was Coordinator to another member of the WSP or to one of the Co-Chairs of the IIJP. This person then becomes the Category Coordinator for the assigned Category.

Sub-Panel Review

  • The WSP breaks into sub-panels.
  • Each sub-panel is responsible for the in-depth review of every Finalist in the categories allocated to each member of the sub-panel
  • For each Category represented in the sub-panel, the following process is followed:
    • The Category Coordinator introduces the Category Finalists
    • Each member of the sub-panel confirms that he/she does not have any real or perceived conflicts of interest in relation to any of the Category Finalists
    • Where a member of the sub-panel has any real or perceived conflicts of interest in relation to any of the Category Finalists, the sub-panel member must remove themselves from all discussion and voting regarding that Category
    • The Category Coordinator leads a discussion of the Category Finalists with regard to the Criteria.
    • The sub-panel may contact a person nominated by the entrant if they feel further information or clarification is required.
    • The sub-panel decides on a recommendation for the Winner, based on the Category Criteria that will be presented to the full panel.

Full Panel Review

  • The full panel convenes to consider the recommendations of each sub-panel and to vote on the Winner of each Category.
  • The final consideration of each Category proceeds as follows:
    • The Category Coordinator introduces the Category Finalists
    • Each member of the full panel confirms that he/she does not have any real or perceived conflicts of interest in relation to the Category Finalists
    • Where a member of the full panel has any real or perceived conflicts of interest in relation to any of the Category Finalists, the full panel member must remove themselves from all discussion and voting regarding the Category
    • The Category Coordinator presents the sub-panel’s assessment of the Finalists, and the recommendation for the Winner of that Category
    • If requested by any member of the WSP or the Co-Chairs of the IIJP, discussion may take place, covering any questions or concerns that the WSP may have regarding the Category Finalists.
    • The Winning Entry is then selected from the Category Finalists by secret ballot in which each member of the WSP casts a single vote nominating their preference for the winner against the Category Criteria.
    • Votes are sealed in an envelope to be counted at a later date using the ‘first past the post’ method. The results are not made known to the members of the IIJP.
    • Should there be a tie, the entry with the highest Moderated Score will be declared the winner.

Part 3: Winner Selection for the State or Territory Award and Australia’s Favourite Event

State or Territory Award – One Step Process

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process
  • National Events (those staged in four or more states or territories) are excluded from this category.
  • Entries do not need to meet the Finalist Threshold Score contribute to the State or Territory Award.The States and Territories of Australia in which each eligible event in the Best Event Categories occurred is determined and points are allocated to those States and Territories as follows:
    • If the entry is not a National Finalist or National Winner: 1 point
    • If the Entry is a National Finalist but not the National Winner: 2 points
    • If the Entry is the National Winner: 3 points
  • Every entry that is not a National Event in the Best Event Category accrues points towards their State or Territory regardless of whether or not they meet the Finalist Threshold Score.
  • The total number of points for each State and Territory is calculated and any State or Territory which does not have at least 7 points including at least one National Finalist and one National Winner is eliminated
  • The total number of points for each of the remaining States and Territories is divided by the population of that State or Territory to yield a State or Territory Score.
  • The State or Territory with the highest State or Territory Score wins the State or Territory Award.
  • For the avoidance of doubt, there are no finalists for this Category.

Australia’s Favourite Event – Public Vote Process

  • Australia’s Favourite Event is the people’s choice Australian Event Award. It is chosen by members of the public from the public events that are finalists in any Best Event Category using the following process:
    • Categories and Criteria remain the same; entrants enter in the normal manner and finalists are selected using the above process
    • Any public events which are finalists in any of the Best Event Categories become the Nominees for Australia’s Favourite Event
    • At the point when Finalists are announced, the Nominees for Australia’s Favourite Event are advised of:
      • their finalist status
      • the day on which public voting for Australia’s Favourite Event commences
      • the day on which the top 4 events nominated for Australia’s Favourite Event will be announced as Finalists
    • On the nominated day, on-line voting for Australia’s Favourite Event will commence on the Australian Event Awards website
    • Any person with an email address can submit a single vote for the Nominee(s) of their choice. The use of voting robots is strictly forbidden and may, at the discretion of the Management Company result in all votes for the Nominee being cancelled.
    • On the nominated day, the top 4 Australia’s Favourite Events will be announced and these events will become the Finalists for Australia’s Favourite Event.
    • Voting will continue for the Finalists until the close of voting which will be at a time set by the Management Company on the day of the Australian Event Awards Ceremony.
    • At the close of voting, the event with the largest number of votes wins the Australian Event Award for Australia’s Favourite Event

Part 4: State and Territory Finalists and Winners

  • Categories and Criteria remain the same; entrants enter in the normal manner and are judged according to the above process
  • All entries in the Best Event Categories are ranked according to Moderated Score.
  • Entries must meet the Finalist Threshold Score to become a State or Territory Finalist or Winner
  • National Events (those staged in four or more states or territories) are not eligible to become a State or Territory Finalist or Winner.For each category, the two highest ranked events that occurred in each state or territory is awarded the State/Territory Finalist status
  • On the Awards night, after the National Finalists and Winners are announced in each category, the State and Territory Winners will be announced and displayed on screen. Certificates for State and Territory Finalists and Winners will be available for collection during the evening

Part 5: Judges’ Special Award

The Judges’ Special Award is not open to nomination or entry from the general public or industry. Only WSP judges are permitted to nominate events, achievements or people.

  1. Only WSP judges on the panel for that year can make nominations for a Judges Special Award.
  2. A judge wishing to make a nomination will be required to provide to the Event Awards, 2 weeks prior to Final Judging Day, an overview of the nomination, for distribution to the full WSP for review prior to Final Judging Day.
  3. On Final Judging Day, the nominating judge(s) will present the nominee to the full WSP. All judges on the panel will be given the opportunity to discuss, vote or (in the case of conflicts of interest) abstain from discussion and voting.
  4. The nomination will require a ‘yes’ vote from 75% of those voting for the award to be conferred.
  5. Should the nomination fail to attain the 75% required, the WSP may, by simple majority on show of hands choose to include the nomination on a holding list to be reconsidered in future years.
  6. The WSP may, at any time, remove a name from the holding list by simple majority on show of hands.
  7. Should a person who is not on the Winner Selection Panel (Public Nominator) contact the Event Awards with the intention of putting forward an event, achievement, person or other matter for consideration for a Judges Special Award, the Event Awards will obtain from that person a short introduction on the subject and will distribute it to all members of the the WSP. If any WSP judge feels that the subject is worthy of a Judges Special Award and wants to take the nomination forward, that judge will liaise with the Public Nominator and develop an overview for the WSP. The judge will provide this to the Event Awards 2 weeks prior to Final Judging Day and the Event Awards will distribute it to the full WSP. The process in steps 3 – 6 above will then be followed in relation to the nomination.

Part 6: Unforeseen Circumstances

  • Should a ruling be required on a matter not covered in this document, the Co-Chairs of the IIJP will make a unanimous ruling on that matter
  • The Co-Chairs of the IIJP may take advice from judges or others as they see fit
  • Generally, resolutions to unforeseen circumstances are documented in the form of a revision to this document ahead of The Event Awards in the following year

Tie-Breaking Rules

  • In the event that two or more candidates have the lowest number of votes, the following procedures take place:
    • If the tied candidates combined have fewer votes than the next highest candidate, the entire tied set are eliminated at once. Their preferences are then redistributed.
    • If the tied candidates combined do not have fewer votes than the next highest candidate, then the candidate with the least number of first preference votes is eliminated. If there is a tie in first preference votes then the candidate with the least number of second preferences is eliminated. This process continues until eighth preferences have been counted
  • If a tie still exists, the candidate with the lowest Moderated Score is eliminated

Part 7: Appeals

  • The Event Awards does not have an appeals process. All decisions by the IIJP are final but the IIJP welcomes feedback and encourages constructive criticism aimed at improving The Event Awards.