S-102 Data Storage

ECC have decided to store the S-102 data at Amazon on an AWS S3 solution. Data is currently stored unencrypted, but an encrypted version is expected to be ready by Q4 2019.
The service is running on an EC2 Ubuntu instance that runs PostgreSQL.

Storage cost considerations

  • Storage cost is evaluated based on terms of quality and service performance.
    • Storage costs are relatively small compared to data transfer costs – 23 USD / 1TB per month.
    • The cost of extra storage, i.e. uncompressed data, can provide benefits as data deliveries are calculated and packaged on the server side. For example, on the server side a calculation is made based on the customer requirements with regards to updates, this is then packaged and delivered directly to the customer. Conversely, the customer has to download all updates, make the calculations on the client side and then extract what is needed – in this instance much more data than is necessary has been transferred at an increased cost to the customer and potentially, the supplier.
    • The recommendation is to make an easier access regime enabling storage of many tiles in advance, which in turn results in an even faster service, at an even lower cost.
    • In addition to the S-102 files, scaled-down GeoTIFF files are stored  for quick use in WMS services.
    • As of September 15, 2017, 36  S-102 datasets were received from the Norwegian Hydrographic Service, a total data volume of 7.71 GB. When compressed as zip files, this dropped to 1.5 GB, which calculates a degree of compression of approx. 80%. The average size per data set typically being 214 MB (uncompressed).
    • As of March 26, 2019, these figures have changed to 45 S-102 data sets totalling 12.21GB, reduced to 2.14 GB after compression. The average size has now increased to 271.4 MB per data set (uncompressed).

In the long term, we will probably store data both compressed and encrypted data, in accordance to the IHO Data Protection scheme.