[OAM-talk] centralized hosting

[Jeffrey Johnson]

>> The initial OAM effort was essentially performed (process & tile
>> imagery) on a single big machine, so yes its quite possible.
>
> Is the source code available somewhere?  How hard would it be for me to set
> up a working prototype (everything but the storage space/processing power)?
> How much work would be involved in modifying the system to work on more than
> one server?  (It wouldn't be too bad since the servers are easily
> partitioned geographically, right?)

Chris Schmidts original code is available here
http://svn.openaerialmap.hypercube.telascience.org/trunk/ or here
http://github.com/ortelius/oam-original ... newer work is here ...
http://github.com/jlivni/openaerialmap

> To scale it up to where it could serve, say, all USGS high resolution
> orthography listed at http://seamless.usgs.gov/products/listofortho.php as
> tiles, what are we talking in terms of number/type of machines, hard drive
> space, and power consumption?
> If we could do that, how much a month could we get during the start-up phase
> in grants and donations?  Enough to pay for the power consumption?

Lets take the NAIP imagery in the US for an example. The raw data for
the entire country is approx 4TB ... a cache of this could be 8-10x
that big (perhaps more). Based on other work I have done processing
datasets like this on EC2, this would take on the order of 50-75k
m1.small compute hours to reproject and cut the tiles. Feel free to
work backward from those numbers.

>> The issue
>> is whether something like this is sustainable over the long term. No
>> matter how much hardware you or anyone else threw at the problem,
>> short of full unfettered access to EC2 with someone footing the bill,
>> we would quickly run up against resource issues ...
>
> Amazon is not the only organization out there who can host a few hundred
> terabytes of data.

But they are the only place where you could 'easily' provision the
machines to process datasets of this size without a massive capital
investment up front. Its not just about storing and hosting ... you
have to get the data from its original format and projection into
tiles suitable for serving.

> And over the long term, surely people and organizations will be willing to
> donate (both monetarily and in-kind) to such a cause.  The need for high
> quality fast access to aerial photography tiles is set to explode with the
> rise of smart phones.

Agreed

Jeff