<HTML>
<HEAD><TITLE>IMO Video Meteor Network</TITLE></HEAD>
<BODY BACKGROUND="metlogo.gif" BGCOLOR="#C0C0C0" BGPROPERTIES="FIXED">
</BODY>
<HR>
<H2>Access to IMO Network Video Meteor Database Archive</H2>
<HR>
<H3>Preface</H3>

<P>Data from the <STRONG>IMO Video Meteor Network</STRONG> is provided via different webservices
<UL>
<LI>Data from individual meteors are provided in PosDat format on this <A TARGET=_blank HREF="http://www.imonet.org/database.html">website</A>. This format is preferred if you like to use tools like RadFind and StrmFind from the <A TARGET=_blank HREF="http://www.metrec.org">MetRec</A> package, or Radiant.
<LI>The same data is also provided for download in CSV format on this <A TARGET=_blank HREF="http://www.meteorflux.org/obs#selectiontab">website</A>. You have a large variety of selection criteria that allow you to download a specific subset of the database. The CSV format provides a flexible interface to import the data in any analysis software.
<LI>Observational statistics from the IMO Network are provided at the same <A TARGET=_blank HREF="http://www.meteorflux.org/obs">website</A>. There you get detailed information about the observing history of our observers, cameras, and the meteor showers we observed.
<LI>Flux data from currently active meteor showers are uploaded in real-time by most cameras to <A TARGET=_blank HREF="http://www.meteorflux.org">MeteorFlux</A>. Flux graphs can be accessed <A TARGET=_blank HREF="http://www.meteorflux.org/rt">here</A> and at the <A TARGET=_blank HREF="http://www.imo.net">IMO Homepage</A>.
<LI>After their quality check, final flux data are made available for analysis <A TARGET=_blank HREF="http://www.meteorflux.org">here</A>.
<LI>The complete IMO network database is archived in the public cloud on <A TARGET=_blank HREF="https://s3.console.aws.amazon.com/s3/buckets/imo-video-meteor-network-archive">AWS S3 Glacier Deep Archive</A>.
</UL>
Here is a desciption on how to retrieve data from the AWS archive.
<HR>
<H3>Archive Structure</H3>
<P>The IMO Video Meteor Database is structured as follows
<UL>
<LI>1993</LI>
<LI>1994</LI>
<LI>...</LI>
<LI>2001</LI>
<UL>
<LI>20010101</LI>
<LI>20010102</LI>
<LI>20010103</LI>
<UL>
<LI>AVIS</LI>
<LI>CARMEN</LI>
<LI>FAMOS</LI>
<UL>
<LI>Logfile</LI>
<LI>Meteor data</LI>
<LI>PosDat files</LI>
<LI>...</LI>
</UL>
<LI>...</LI>
<LI>VK1</LI>
</UL>
<LI>20010104</LI>
<LI>...</LI>
<LI>20011231</LI>
</UL>
<LI>2002</LI>
<LI>...</LI>
<LI>2019</LI>
</UL>

<HR>
<H3>Access to the IMO Network Database Archive</H3>
<P>S3 Glacier is a cost efficient archive solution offered by Amazon Web Services (AWS). Glacier Deep Archive is the preferred storage class for long-term archiving, were data are accessed in rare cases only. So this archive is not meant for every-day access, but as a backup solution for our database.
<P>Files in Glacier Deep Archive are not immediately accessible. They have to be restored first, which takes typically about one day, before thay can be downloaded. The process is as follows:
<UL>
<LI>Access the IMO network archive at <A TARGET=_blank HREF="https://s3.console.aws.amazon.com/s3/buckets/imo-video-meteor-network-archive">https://s3.console.aws.amazon.com/s3/buckets/imo-video-meteor-network-archive</A>. Login with the account ID and the username given in the screenshot. The password will be provided to you on request.<BR>
<IMG SRC="glacier1.png" ALT="Login">
<LI><P>You are then using the S3 console to navigate though the archive. The top level structure are the years as described in the archive structure.<BR>
<IMG SRC="glacier2.png" ALT="S3 Console">
<LI><P>All data from a single day are compressed in one ZIP file.<BR>
<IMG SRC="glacier3.png" ALT="Single Day">
<LI><P>Click on the date you are interested, and you will see the file details. The Download button is inactive, as files in Glacier Deep Archive need to be restored first. This can be achieved by clicking on "Initiate Restore"<BR>
<IMG SRC="glacier4.png" ALT="Initiate Restore">
<LI><P>In the next screen you have to specify, how long the restored data shall be made available. As restores are charged, you should select "standard retrieval" and a reasonable duration, e.g. 14 days.<BR>
<IMG SRC="glacier5.png" ALT="Restore Parameters">
<LI><P>Once the restore is initiatied, you need to wait for about a days.<BR>
<IMG SRC="glacier6.png" ALT="Restore Stared">
<LI><P>If you login the next day and browse to the same ZIP file, the download button is activated.<BR>
<IMG SRC="glacier7.png" ALT="Restore Finished">
<LI><P>Click on it to download the files to your computer. After the specified time, the file will be deleted and another restore needs to be initiated if you want to download the file again.<BR>
<IMG SRC="glacier8.png" ALT="Download">
<LI><P>If you want to downloaded several nights at once, select all relevant files and initiate the restore via the "Action" menu.<BR>
<IMG SRC="glacier9.png" ALT="Bulk Restore">
<LI><P>"Bulk retrieval" is the right option in this case. It takes a little longer, but it provides all data at once.<BR>
<IMG SRC="glacier10.png" ALT="Bulk Restore Parameters">
<LI><P>In order down download multiple files at once, you have to start the <A TARGET=_blank HREF="https://eu-central-1.console.aws.amazon.com/cloudshell?region=eu-central-1">Amazon cloud shell</A>. Sign in with the same credentials as to the S3 console.<BR>
<IMG SRC="glacier1.png" ALT="Login">
<LI><P>You can then copy all or only selected restored files into the cloud shell with the aws s3 cp command, e.g. <EM>aws s3 cp --force-glacier-transfer s3://imo-video-meteor-network-archive . --recursive --exclude "*" --include "1999/199903*"</EM><BR>
<IMG SRC="glacier11.png" ALT="Login">
<LI><P>Since you cannot download multiple files to your client, you need to create a single  file from all the copied files with the zip command, e.g. <EM>zip download.zip 1999*.zip</EM><BR>
<IMG SRC="glacier12.png" ALT="Login">
<LI><P>Finally you can download the files via the Actions menu<BR>
<IMG SRC="glacier13.png" ALT="Login"><BR><BR>
<IMG SRC="glacier14.png" ALT="Login">
<LI><P>Do not forget to delete the files from the cloud shell, e.g. with <EM>rm -rf 1999 download.zip</EM><BR>
<IMG SRC="glacier15.png" ALT="Login">
</UL>
<HR>
<P>
<ADDRESS><A HREF="mailto:webmaster@imonet.org">webmaster@imonet.org</A>; last change: January 20, 2023</ADDRESS>
</BODY>
</HTML>