MT: Calculating METEOR Score of two different files

Question

For an Machine Translation evaluation I need to calculate the METEOR score between the translation output file and the reference file.

I already found this question How can I implement meteor score when evaluating a model when using the meteor_score module from nltk?, but it was not helpful.

• Is there any official Python package or Github repository for the calculation of the METEOR score you would recommend?
• How to calculate the METEOR between the reference file and the output translation file?

Answer 1

I found out you have to download METEOR from https://www.cs.cmu.edu/~alavie/METEOR/index.html#Download .

For calculating the METEOR Score you need Java as well.

Run the following command to compare the reference and the test file and to get the METEOR score:

java -Xmx2G -jar meteor-1.5.jar test-file reference-file -l de

This page is well documented: https://www.cs.cmu.edu/~alavie/METEOR/README.html