WALDO

UPDATE July 18 2024: We are beginning to upload broadband data from the first four months of 2023. It should be finished within a few days. After that, we will finally shift our attention back to narrowband data, putting in more recent data and beginning to catch it up.

UPDATE June 30 2024: Due to recent bot/hacker activity, a (totally free) account is now required in order to download data. Click “create account” above, or login with an existing account. Without an account, however, you can still browse what is available, but the download button will be missing.

UPDATE: June 3 2024: Due to some hacker/bot activity, we are experimenting with having accounts required in order to download data. All data on WALDO will remain free, no permission required in order to set up an account or download anything. It will just be required that you confirm your account with an email address.

UPDATE: January 3 2024: We have added broadband data through December 2022.

UPDATE: November 12  2023: We have added broadband data through December 2021

UPDATE: June 12 2023: Lot more broadband data has been added covering through September 2021, and more is coming soon. We’re also adding a lot of hi-resolution narrowband data, alphabetically by site name. In addition, WALDO World is now incorporated as a nonprofit corporation in the State of Georgia, so that we may continue our mission to collect, curate, and distribute ELF/VLF/LF radio data for scientific purposes, and general open public use.

UPDATE July 13 2022: Broadband VLF and LF data from 2020 have been added. We are moving on very soon to a large volume of narrowband data, which are smaller in size but have a lot of value.

UPDATE October 17 2021: Broadband VLF and LF data from 2018 and 2019 have been added, totaling 100s of TB. In addition, a large amount of new data from the Siple Station experiment have been added from 1973-1988.

UPDATE October 15 2021: WALDO now has doi’s so you can cite any data from WALDO. The broadband VLF data has doi 10.48322/fzcf-f191. The broadband LF data has doi 10.48322/fwte-dv13. The narrowband data has doi 10.48322/6t7b-s324. Our thanks to the SPASE team at NASA Goddard for helping set these up with dois and metadata!

UPDATE February 24 2021: WALDO is now using secure SSL encryption, so all links have been changed from http://XXX to https://XXX. Things should all be working exactly the same, but please contact us if that is not the case.

UPDATE January 25 2021: WALDO has officially joined NASAs “Virtual Wave Observatory” alongside other ground and satellite instrument datasets, and is now searchable via VWO. Out thanks to Chuck Dolan and Shing Fung for setting that up. To search for WALDO data within VWO, go to https://vwo.nasa.gov, click on Data Query, enter in a time range of interest, select “WALDO” within on the observatory box, and then if it isn’t greyed our, click on either “AWESOME LF” or”AWESOME VLF” under the instruments box. You can then click on “Apply the above conditions”, and then “Run Query”. In the Query builder that pops up, click on “Expand”, and then download the files. We also have a search capability here on WALDO, but this may be useful if you want to correspond WALDO data with other datasets in VWO.

UPDATE November 28 2020: Progress in getting data onto WALDO continues at a steady pace. The past couple months we moved all Georgia Tech broadband broadband data from the year 2017. This took a long time because it included a large amount of data at the “LF” sampling rate in the days before and after the August 21 solar eclipse, and these files are 10 times larger (ordinarily we only save snippets at the LF rate, but kept continuous data around the eclipse).  But it’s all up now. We are moving on next to post some broadband data from Adelaide, Australia, and after that will start porting over data from Chistochina, Alaska, for the years 2004-2006. Remember, if there’s any specific data you believe may have been captured and you want us to prioritize posting, please let us know, we may be able to help.

UPDATE November 6 2020: In the past several updates, most prominently August 8 2020, we mentioned a couple of scripts that processed narrowband data from our server at Georgia Tech and posted it on WALDO. This took several months but is finally complete. This means now that a huge amount of narrowband data is on WALDO. The vast majority of it prior to 2016 is low-resolution 1-Hz data, which is suitable for most purposes, but we do have the wheels in motion to get the 50-Hz narrowband data up on WALDO, as well. This is going to take some time, though, and isn’t priority #1. Most of this data is still stuck on an old server at Stanford that has been repaired, and we are having that data transferred to hard drives and then mailed to Georgia Tech, at which point we will begin posting it online. I expect, due to sheer number of files which slows things down, that will be a monthlong process, but updates will continue to be here.

UPDATE August 25 2020: Very excited about two new additions to WALDO. First, we have added a set of python tools which have been created and made available publicly, with thanks to David Richardson, a PhD student at Georgia Tech. These can be found under the “scripts” page. Second, we have added a Google map with pins showing all the sites available on WALDO. If you click on a pin, it will show how much data are available for each year, for each type of data (broadband, narrowband, LF). The map is in the main navigation menu at the top. The map has to be manually updated, and it reflects only the data currently on WALDO, not the data that will eventually be on WALDO. The map does have to manually updated so it’s possible there will be some data just recently added that is not yet reflected on the map.

UPDATE August 8 2020: We continue to transfer a significant amount of narrowband data. There are two scripts running, one which is working from A to Z alphabetically by site name, and is currently on “C”, and another which is working from Z to A, and is currently on “L”. At some point the two scripts will meet. This will cover all narrowband data collected until early July 2017.  In addition, we are also now transferring Georgia Tech VLF and LF from the first half of 2017 on to WALDO. Finally, we have added a large amount of data from a CU-Denver receiver in Denver.

UPDATE May 20 2020: This is a quick update, we just added a new searchbar page! This will make it easier to find files from a certain date, for example.

UPDATE May 13 2020: What better way to shelter-in-place than to cozy up with some cool VLF data? Luckily, WALDO is here to give you your fix. A couple of updates: (1) AGU’s EOS magazine has featured an article about WALDO, which you can read here. (2) We have completed uploading Palmer Station data from 2000-2013. We have two priorities for next steps. The first is Narrowband data. We have two scripts running, one working alphabetically by site name, transferring and processing data from an internal Georgia Tech server to WALDO. The other script is the same but working in reverse alphabetical order. Eventually they will meet in the middle. Our second priority is Siple Station experiment data. We’ve only put up some 1986 data, and a lot more is currently being rescued from a data server at Stanford University. Hopefully that will start to go up soon.

UPDATE December 31 2019: Happy new year! WALDO is now uploading narrowband data from many sites around the world. There are actually two scripts running which transfer data from an existing Georgia Tech server to WALDO, one is working  alphabetically forwards from A, the other alphabetically backwards from Z. At some point weeks down the road, those scripts will meet at “the golden spike” (thanks, Leland). Our aim is to upload everything we have from 2016 and earlier. Palmer station broadband data is also being uploaded, but at a slower pace, right now we have broadband data from May 2000 through March 2010, and we are uploading that chronologically.

UPDATE December 9 2019: WALDO is now officially announced! We are continuing to post data almost 24-7. Recently we have uploaded Palmer Station broadband from 2000-2009 and working our way through 2010 now. We have also uploaded the first data from the Siple Station data. We started off with 1986 data from Lake Mistissini (Canada), and there is more coming. Our priorities for data to upload next are: (1) More data from Siple Station and Lake Mistissini, (2) South Pole broadband VLF data through the 2000s, (3) IHY/ISWI/AWESOME narrowband data from 2007-2010. There’s a lot more coming after that so check back here for updates. If there’s specific data you believe was captured that hasn’t yet been posted, and you’d like us to prioritize something, please let us know.

UPDATE November 13 2019: We have now blown well past 100 TB and are coming up on the 150 mark. We finished transferring all Georgia Tech broadband data from 2016 and earlier, and then moved a large amount of old Stanford broadband data from 2007-2009. This includes many broadband sites in Alaska including conjunction with HAARP experiments during those years. Then, we moved on to Palmer Station data, one year at a time. We have several complete years (2000, 2007. 2008), and are moving through things as quick as we can. Within a month or so we may be finished with Palmer Station covering May 2000 through 2013.

UPDATE October 2 2019: Though we still haven’t officially announced WALDO, you might have reached this page having noticed our AGU abstract and Googling around. If so, hi! Yep, we’re here. So here’s an update: in late August we decided to pause transferring the old Stanford data, and start transferring some more recent Georgia Tech data, including a lot of LF data at 1 MHz sampling frequency. And that is continuing in earnest. Our goal is to transfer all broadband data collected in 2016 and earlier by Georgia Tech. We are currently finished 2014 and 2015 and we are about halfway through 2016. By the middle/end of October, we anticipate finishing 2016, at which point we will shift back to transferring old Stanford data again, focusing on broadband. When we make enough progress on that, we’ll come back to Georgia Tech data and transfer 2017 data. Somewhere in that process we’ll begin adding narrowband data, as well, but we’ll try to do this in parallel with the broadband.

We’re sitting at around 75 TB of data now on WALDO, which sounds like a lot but it is still is a drop in the bucket compared to the 1000 TB or so we expect within a year. By AGU meeting this year, we should be in the 200 TB range, enough to announce WALDO while it continues to ramp up. We’re moving about 1.5-2 TB per day, but are looking for ways to increase that transfer rate. Here’s a plot that shows our WALDO storage as a function of time.

UPDATE August 9 2019: We have crossed the 10 TB barrier, and things are starting to roll faster now. We’re at 500-1000 GB per day of uploading. We are focusing on broadband data, including some Palmer Station (Antarctica), multiple sites in Alaska (Chistochina, Juneau), and some more recent (2014 and 2015) Georgia Tech data from sites in Georgia, including some data with 1 MHz sampling frequency.

We also uploaded broadband data from a Japan receiver that was <100 km from the March 2011 Tohoku earthquake, that was featured in this paper. The receiver operated in the hours before the earthquake and even for the first minute or two once it started. We also have narrowband data from the months leading up to that day, which we will upload later when we shift gears more to narrowband data.

UPDATE June 24 2019: We are now in the process of setting up WALDO and are making good progress. Data currently resides in one of three places: On an old server at Stanford, and in notebooks of DVDs (roughly 80,000 DVDs in total) at either Georgia Tech or CU-Denver. As of June 2019, the infrastructure for the database has been set up, via a Google Drive account. Both Georgia Tech and CU-Denver have machines that can read in a stack of DVDs in one shot, and the data from the old Stanford server is being rescued and placed into a different Google Drive. A script is now working to then take these data, sort them on WALDO, and automatically generate the quick-look plots. Formats of the files also have to be converted since the recording software used several different formats over the years, but we will be converting everything to have the same format. As data are uploaded, everything is updated on this website automatically, nothing is being embargoed as things get transferred save for delays uploading to the cloud. Some modest amounts of data have been uploaded. However, it will take some time to work out small bugs as we expand the database to more sites. Our hope is to take the summer to finish off all remaining bugs and issues, then have an hourly technician hire do the work of loading the data from there, a process which unfortunately will take months (80,000 DVDs plus a server with 100s of TB!).