West Australia's advanced computing hub, iVec, has used a million-dollar grant to acquire a Sun StorageTek 6540 disk storage array and L8500 Tape Library, which it will put to work as a shared storage pool that researchers can use to store and access large data sets.
Sun's robotic storage and SAM-FS were the key to the deal, as together they will allow iVec to tier its data across all of its new equipment but still provide rapid access to information.
"We already have three high performance computers," explains iVec CEO Andrew Rohl. "They all have local disk and today you do your calculations and move the data away. It becomes long-term storage that is shoved into a cupboard."
iVec's new arrangements will see data stored on a blend of disk and nearline tape.
"One project we are considering will use LandSat data for all of Western Australia from a 30-year archive," Rohl says. The new system will see metadata about this dataset store on Sun servers accompanying the new server, with this layer created by the SAM-FS software.
Queries will result in the robotic tape unit retrieving the appropriate data.
"With the new system this kind of request will be a one-minute query," Rohl says. "The very large images will initially be stored offline and then brought to nearline," and all data will be accessible over iVec's network to allow remote researchers to access data.
Rohl says the new system will be transparent to users, but expects some modification and customisation by developers will be needed to wrestle datasets into place and help create interfaces for researchers.
The agency's initial purchase includes 80 terabytes of disk and 400 terabytes of tape, but the tape robot has capacity for 2 petabytes of data. The robot has capacity for 4 petabytes of data, scalability that Rohl says is important for the agency's future needs.
At least one aspect of the new rig, however, has appeal for its use of older technologies.
"We like the fact data is stored as tar files," Rohl says. "We are pretty confident that we will always be able to read tar files!"