Provide a REST API for BlockSci

[timothyylim]

I'm interested in running BlockSci as a web service where users can run queries against the python interface through a REST API. I have a few questions regarding this:

1. How does the provided AMI keep in sync with the network? Would it be sufficient to run the AMI and spin up a flask server with BlockSci installed?

2. If I were to install BlockSci alongside a Core Node, would I need 60GB worth of RAM to parse it whenever a new block comes in? Or is that just necessary to build the DB at the beginning?

Thank you!

[bfxtsales]

Terrific idea. If I understand correctly, a client/server architecture where the AWS image operates in a server role supporting multiple simultaneous clients? Would the C++ helper functions execute at the client or the server? Do we know if BlockSci was designed with this kind of client/server model in mind?

[hkalodner]

@timothyylim

Previous versions of the AMI only contained static versions of the blockchain. However with BlockSci v0.4 the AMI performs hourly updates to the latest chain.

Incremental updates using the parser use somewhere between 10 and 15 GB of memory while in progress and take about a minute and a half when run once an hour.

The 60 GB recommendation is to be able to perform updates while also having all of the transaction data loaded into memory by the analysis library.

[hkalodner]

@bfxtsales
BlockSci was mainly designed for our Jupyter Notebook access model with a single server running a BlockSci instance and many users simultaneously performing analysis interactively in python sessions on that server.

BlockSci uses memory mapping in order to achieve its rapid performance and thus needs local access to the blocksci_parser output data.

[hkalodner]

I definitely like the idea of building a REST API on top of BlockSci to provide another method of working with it.

I wont have time to work on it myself, but it would be great if someone wanted to pick up the reigns and work on it. It would mainly consist of implementing a thin REST API layer on top of BlockSci which would make calls to the BlockSci library and return the results to it's clients.

[archienorman11]

Has anyone made any progress here, if not, is anyone interested in collaborating?

[aknirmal90]

I'm interested in getting this done.

@hkalodner
i) Does the incremental update (write-op) interfere with a read-op if both operations occur at the same time / are concurrent?

@bfxtsales
BlockSci was mainly designed for our Jupyter Notebook access model with a single server running a BlockSci instance and many users simultaneously performing analysis interactively in python sessions on that server.

ii) Are multiple concurrent read-operations supported? From your previous comment, I would tend to think the answer to this question is yes.