"list index out of range" error -- need to update script for Downloads API v3?

[jlg89]

This is on macOS 10.13.6 (old, I know), Python 3.9.23 (via MacPorts). Been working previously. Not sure when it started throwing the "list index out of range" error.

+==========================================================================+
|     _____                              __  __          __      __        |
|    / ___/___  ______   _____  _____   / / / /___  ____/ /___ _/ /____    |
|    \__ \/ _ \/ ___/ | / / _ \/ ___/  / / / / __ \/ __  / __ `/ __/ _ \   |
|   ___/ /  __/ /   | |/ /  __/ /     / /_/ / /_/ / /_/ / /_/ / /_/  __/   |
|  /____/\___/_/    |___/\___/_/      \____/ .___/\__,_/\__,_/\__/\___/    |
|                                         /_/                              |
+==========================================================================+

[PaperMC Server Updater]
[Handles the checking, downloading, and installation of server versions]
[Written by: Owen Cochell]

# Loading configuration data from file [/usr/local/minecraft/version_history.json] ...

Server Version Information:
  > Version: [1.21.6]
  > Build: [28]
[ --== Checking For New Version: ==-- ]

Loading version information ...
# Comparing local <> remote server versions ...
# No new version available.
# Loading build information ...
# Comparing local <> remote builds ...
# New build available! - [Build: 46]

[ --== Version check complete! ==-- ]

[ --== Version Selection: ==-- ]

# Loading version information ...
# Selecting latest version - [1.21.6] ...
# Loading build information ...
# Selecting latest build - [46] ...

You have selected:
   > Version: [1.21.6]
   > Build: [46]

[ --== Version Selection Complete! ==-- ]

# Creating temporary directory...
# Temporary directory created at: /tmp/tmp9x_ji9ww

[ --== Starting Download: ==-- ]


+==================================================+
> !ATTENTION! >
An error occurred during a request operation.
Fail Point: File Download
Your check/update operation will be canceled.
Detailed error info below:
+==================================================+
  [ --== The Following Error Has Occurred: ==-- ]
+==================================================+
Error Name: list index out of range
+==================================================+
Full Traceback:
Traceback (most recent call last):
  File "/usr/local/minecraft/./server_update.py", line 1523, in get_new
    path = self.update.download_file(self.fileutil.temp.name, ver, build_num=build, call=progress_bar, check=self.integrity)
  File "/usr/local/minecraft/./server_update.py", line 525, in download_file
    path = os.path.join(path, data.getheader('content-disposition', default='').split("''")[1])
IndexError: list index out of range
+==================================================+
(Can you make anything of this?)
Please check the github page for more info: https://github.com/Owen-Cochell/PaperMC-Update.

[jlg89]

I was able to manually download the latest paper.jar using curl, and manually updated version_history.json accordingly. The server_update.py script now runs successfully since it's not trying to download anything. Let me know what I can do to provide more troubleshooting data.

[jlg89]

OK, they've changed the way updates occur. Grr.

https://docs.papermc.io/misc/downloads-api/

[jlg89]

Updated python to 3.13.5 just for argument's sake.

Checking for updates...
+==========================================================================+
|     _____                              __  __          __      __        |
|    / ___/___  ______   _____  _____   / / / /___  ____/ /___ _/ /____    |
|    \__ \/ _ \/ ___/ | / / _ \/ ___/  / / / / __ \/ __  / __ `/ __/ _ \   |
|   ___/ /  __/ /   | |/ /  __/ /     / /_/ / /_/ / /_/ / /_/ / /_/  __/   |
|  /____/\___/_/    |___/\___/_/      \____/ .___/\__,_/\__,_/\__/\___/    |
|                                         /_/                              |
+==========================================================================+

[PaperMC Server Updater]
[Handles the checking, downloading, and installation of server versions]
[Written by: Owen Cochell]

# Loading configuration data from file [/usr/local/minecraft/version_history.json] ...

Server Version Information:
  > Version: [1.21.6]
  > Build: [46]
[ --== Checking For New Version: ==-- ]

Loading version information ...
# Comparing local <> remote server versions ...
# New Version available! - [Version: 1.21.7]

[ --== Version check complete! ==-- ]

[ --== Version Selection: ==-- ]

# Loading version information ...
# Selecting latest version - [1.21.7] ...
# Loading build information ...
# Selecting latest build - [17] ...

You have selected:
   > Version: [1.21.7]
   > Build: [17]

[ --== Version Selection Complete! ==-- ]

# Creating temporary directory...
# Temporary directory created at: /tmp/tmpy3fgzfn3

[ --== Starting Download: ==-- ]


+==================================================+
> !ATTENTION! >
An error occurred during a request operation.
Fail Point: File Download
Your check/update operation will be canceled.
Detailed error info below:
+==================================================+
  [ --== The Following Error Has Occurred: ==-- ]
+==================================================+
Error Name: list index out of range
+==================================================+
Full Traceback:
Traceback (most recent call last):
  File "/usr/local/minecraft/./server_update.py", line 1523, in get_new
    path = self.update.download_file(self.fileutil.temp.name, ver, build_num=build, call=progress_bar, check=self.integrity)
  File "/usr/local/minecraft/./server_update.py", line 525, in download_file
    path = os.path.join(path, data.getheader('content-disposition', default='').split("''")[1])
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
+==================================================+
(Can you make anything of this?)
Please check the github page for more info: https://github.com/Owen-Cochell/PaperMC-Update.

[jlg89]

I wrote an update shell script based on the current Downloads API documentation. It requires jq as mentioned there.

I used embedded if statements without any exits so you can incorporate stopping/starting your minecraft server at the top & bottom of the script.

Use at your own risk, YMMV, it works for me, they strongly recommend against auto-updating, etc.

#!/bin/sh
CURVER=`cat current_paper_version.txt`
PROJECT="paper"
USER_AGENT="something_unique/1.0.0 (you@domain.com)"
echo "Checking for paper update..."
MINECRAFT_VERSION=$(curl -s -H "User-Agent: $USER_AGENT" https://fill.papermc.io/v3/projects/${PROJECT} | jq -r '.versions | to_entries[0] | .value[0]')
LATEST_BUILD=$(curl -s -H "User-Agent: $USER_AGENT" https://fill.papermc.io/v3/projects/${PROJECT}/versions/${MINECRAFT_VERSION}/builds | jq -r 'map(select(.channel == "STABLE")) | .[0] | .id')
if [ "$LATEST_BUILD" != "null" ]; then
  NEWVER="$MINECRAFT_VERSION"."$LATEST_BUILD"
  echo "Current: $CURVER"
  echo "Latest: $NEWVER"
  if [ "$CURVER" != "$NEWVER" ]; then
    echo "Downloading $NEWVER"
    # First check if the version exists
    VERSION_CHECK=$(curl -s -H "User-Agent: $USER_AGENT" https://fill.papermc.io/v3/projects/${PROJECT}/versions/${MINECRAFT_VERSION}/builds)
    # Check if the API returned an error
    if echo "$VERSION_CHECK" | jq -e '.ok == false' > /dev/null 2>&1; then
      ERROR_MSG=$(echo "$VERSION_CHECK" | jq -r '.message // "Unknown error"')
      echo "Error: $ERROR_MSG"
    else
      # Get the download URL directly, or null if no stable build exists
      PAPERMC_URL=$(curl -s -H "User-Agent: $USER_AGENT" https://fill.papermc.io/v3/projects/${PROJECT}/versions/${MINECRAFT_VERSION}/builds | jq -r 'first(.[] | select(.channel == "STABLE") | .downloads."server:default".url) // "null"')
      if [ "$PAPERMC_URL" != "null" ]; then
        # Save the current version
        cp paper.jar paper.jar.old
        # Download the latest Paper version
        curl -o paper.jar $PAPERMC_URL
        echo "Download completed"
        echo $NEWVER > current_paper_version.txt
      else
        echo "No stable build for version $MINECRAFT_VERSION found :("
      fi
    fi
  fi
fi
echo "Done."

[OwenCochell]

Hello!

Sorry for the very large response delay, things have been busy for me recently. This is indeed related to the new V3 API the paper team rolled out.

I have just pushed some fixes that now use the new API, as well as some minor improvements under the hood. This script is also now compliant with the new user agent requirements outlined in the V3 documentation.

Please let me know if you run into any issues! I believe I have tested everything but without a proper unit testing framework (a future goal) it's hard to know for sure. I apologize once again for the delay, things have been busy as of late and my response time has been poor (I have still have a pending PR for this project I need to address...). I'll be closing this issue in a few days if there is no extra activity.

Thanks for reporting this, it would have flown under my radar otherwise!

[jlg89]

Just ran the updated updater. It downloaded 1.21.7-2 (the first 1.21.7 release) instead of 1.21.7-26 (the latest 1.21.7 release).

+==========================================================================+
|     _____                              __  __          __      __        |
|    / ___/___  ______   _____  _____   / / / /___  ____/ /___ _/ /____    |
|    \__ \/ _ \/ ___/ | / / _ \/ ___/  / / / / __ \/ __  / __ `/ __/ _ \   |
|   ___/ /  __/ /   | |/ /  __/ /     / /_/ / /_/ / /_/ / /_/ / /_/  __/   |
|  /____/\___/_/    |___/\___/_/      \____/ .___/\__,_/\__,_/\__/\___/    |
|                                         /_/                              |
+==========================================================================+

[PaperMC Server Updater]
[Handles the checking, downloading, and installation of server versions]
[Written by: Owen Cochell]

# Loading configuration data from file [/usr/local/minecraft/version_history.json] ...

Server Version Information:
  > Version: [1.21.7]
  > Build: [21]
[ --== Checking For New Version: ==-- ]

Loading version information ...
# Comparing local <> remote server versions ...
# No new version available.
# Loading build information ...
# Comparing local <> remote builds ...
# New build available! - [Build: 2]

[ --== Version check complete! ==-- ]

[ --== Version Selection: ==-- ]

# Loading version information ...
# Selecting latest version - [1.21.7] ...
# Loading build information ...
# Selecting latest build - [2] ...

You have selected:
   > Version: [1.21.7]
   > Build: [2]

[ --== Version Selection Complete! ==-- ]

# Creating temporary directory...
# Temporary directory created at: /tmp/tmpc3jnf42m

[ --== Starting Download: ==-- ]

Downloading:[############################################################] 51790805/51790805
# Integrity test passed!
# Saved file to: /tmp/tmpc3jnf42m/paper-1.21.7-2.jar

[ --== Download Complete! ==-- ]

[ --== Installation: ==-- ]

# Creating backup of previous installation ...
# Backup created at: /tmp/tmpc3jnf42m/backup
# Deleting current file at /usr/local/minecraft/paper.jar ...
# Removed original file!
# Copying download data to root directory ...
# (/tmp/tmpc3jnf42m/paper-1.21.7-2.jar > /usr/local/minecraft/paper.jar)
# Done copying download data to root directory!

[ --== Installation complete! ==-- ]

# Cleaning up temporary directory...
# Done cleaning temporary directory!

Update complete!

I ran my shell script immediately afterward, and it downloaded 1.21.7-26. (It says "current" is 1.21.7-21 because that's the last version it downloaded).

Checking for updates...
Current: 1.21.7.21
Latest: 1.21.7.26
Downloading 1.21.7.26
Saving previous version
Downloading new version
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 50.3M  100 50.3M    0     0  10.8M      0  0:00:04  0:00:04 --:--:-- 10.8M
Download completed

[OwenCochell]

Hello,

Thanks for pointing that out. Should hopefully be fixed now, version 3.0.1 is what you want. Let me know if you run into any trouble!

Thanks!

[jlg89]

Ran 3.0.1, though it needs to be published, 3.0.0 is still the current release. Looks like 3.0.0 might have written the correct (latest) version to version_history.json but downloaded the oldest version (can't verify this, I might have done it myself while troubleshooting prior to 3.0.0, slept since then). Not a problem either way in my case, since I did actually download the latest version.

+==========================================================================+
|     _____                              __  __          __      __        |
|    / ___/___  ______   _____  _____   / / / /___  ____/ /___ _/ /____    |
|    \__ \/ _ \/ ___/ | / / _ \/ ___/  / / / / __ \/ __  / __ `/ __/ _ \   |
|   ___/ /  __/ /   | |/ /  __/ /     / /_/ / /_/ / /_/ / /_/ / /_/  __/   |
|  /____/\___/_/    |___/\___/_/      \____/ .___/\__,_/\__,_/\__/\___/    |
|                                         /_/                              |
+==========================================================================+

[PaperMC Server Updater]
[Handles the checking, downloading, and installation of server versions]
[Written by: Owen Cochell]

# Loading configuration data from file [/usr/local/minecraft/version_history.json] ...

Server Version Information:
  > Version: [1.21.7]
  > Build: [26]
[ --== Checking For New Version: ==-- ]

Loading version information ...
# Comparing local <> remote server versions ...
# No new version available.
# Loading build information ...
# Comparing local <> remote builds ...
# No new builds found.

[ --== Version check complete! ==-- ]

Contents of version_history.json (after running 3.0.1):

{"oldVersion":"1.21.7-2-4045339 (MC: 1.21.7)","currentVersion":"1.21.7-26-533d93c (MC: 1.21.7)"}