ESO is Using a New System to Allocate Telescope Time. It’s Working Well

Most astronomers know the struggle of getting time on the world’s most powerful telescopes. Even though this observing time might literally be the most important thing to their career prospects, there are always more studies than there is time available to perform them. Typically, each telescope system has a panel of experts that determine which proposals will get observational time and which won’t. However, the European Southern Observatory (ESO), based in Germany but with observational telescopes in Chile, decided to try a new proposal review method – peer review.

Peer review is a common process in academic circles. Most papers published in academic journals are subjected to it before publication. In peer review processes, other experts in the field of study being explored in the paper provide comments to the authors, in some cases accepting the manuscript for publication and in some cases asking for further revisions or rejecting it outright.

ESO has taken that model and applied it to its selection of proposed observing times. To start, they only took about 20% of the observational time in the telescope and subjected it to the new method while leaving the other 80% in the hands of its more traditional Observing Programmes Committee, which is structured like a typical review committee. Most of those times were for studies that required more than 16 hours of observational time.

That left 20% of the time for studies that were less than 16 hours and which were to take place during a period known as P110 and P111. P110 lasted from last October to this March, and P111 is currently ongoing and will end in September. It was subjected to what is known as the Distributed Peer Review (DPR) process. In this process, other researchers who submitted proposals for the same block of time were also required to review competing proposals.

Each proposal a researcher submitted required them to provide a peer review for ten other proposals. For P110 alone, there were 435 proposals submitted by 379 different individuals, meaning some researchers submitted more than one proposal into the DPR system. 

It would be a pretty useless system if the reviewers didn’t understand what they were reviewing. So the DPR team utilized a new tool at ESO that requires researchers to list their scientific expertise via keywords when they are submitting a proposal. They could then use these keywords to match individual reviewers with proposals in their designated field of expertise.

To measure the success of this system, there is a simple metric to use, and again one familiar to anyone who has been through a peer review process before – how useful were the peer review comments? The authors of a new paper exploring the outcome of the DPR process utilized feedback from the researchers on how helpful the comments their proposals received were. On almost every metric, the DPR reviews provided better feedback than the ones obtained from the committee.

With that feedback, ESO plans to continue the DPR review process in upcoming observational slots. It might even prove to be a model for other observatories in the future and help ease the burden of observational committees when making some of the most critical decisions in astronomy.

Learn More:
Jerabkova et al – The First Results of Distributed Peer Review at ESO Show Promising Outcomes
UT – Need a Project? You can Build a Paper Model of the Extremely Large Telescope
UT – Images of 42 of the Biggest Asteroids in the Solar System
UT – A LEGO® Version of the Very Large Telescope. It Even has a Laser Interferometer

Lead Image:
Calibration lasers on the Very Large Telescope at ESO in Chile.
Credit – ESO / A. Ghizzi Panizza