Query return empty list

[jeanlucmongrain]

https://github.com/haadcode/orbit-db-eventstore#orbit-db-eventstore

An append-only log with traversable history. Useful for "latest N" use cases or as a message queue.

everything I tested with, including ipfs/go-ipfs is on master.

package main

import (
	"log"
	"time"
	"fmt"
	"flag"
	"io"

	"github.com/keks/go-orbitdb"
	"github.com/keks/go-ipfs-colog"
)

func main() {
	topic := flag.String("topic", "TOPICNAMEYEAH", "topic name")
	list := flag.Bool("list", false, "List entries")
	flag.Parse()

	db, err := orbitdb.NewOrbitDB(*topic)
	if err != nil {
		log.Fatal(err)
	}
	es := orbitdb.NewEventStore(db)

	if !*list {
		data := time.Now().UTC().String()
		fmt.Printf("Add entry %q\n", data)
		_, err = es.Add(data)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Println("Added entry")
		return
	}

	fmt.Println("Query:")
	result := es.Query(colog.Query{})
	for {
		event, err := result()
		if err == io.EOF {
			break
		}
		if err != nil {
			fmt.Printf("Error: %s\n", err.Error())
			continue
		}
		var s string
		if err = event.Get(&s); err != nil {
			fmt.Println(err.Error())
		}
		fmt.Printf("Event: %q\n", s)
	}
	fmt.Println("End of events")
}

$ /tmp/test
Add entry "2016-12-26 11:05:02.85888944 +0000 UTC"
Added entry
$ /tmp/test
Add entry "2016-12-26 11:05:03.714787565 +0000 UTC"
Added entry
$ /tmp/test
Add entry "2016-12-26 11:05:04.138817311 +0000 UTC"
Added entry
$ /tmp/test
Add entry "2016-12-26 11:05:05.247096986 +0000 UTC"
Added entry
$ /tmp/test
Add entry "2016-12-26 11:05:05.685892599 +0000 UTC"
Added entry
$ /tmp/test
Add entry "2016-12-26 11:05:06.020461051 +0000 UTC"
Added entry

Nothing else was listening to that topic.

$ /tmp/test -list
Query:
End of events

empty...

I only get result for query, when I query in the same process that produced events. And I only get the events that came from that process. not those from other producers.

Is the problem query logic itself? or is EventStore that is not persistent?

[keks]

Sorry I missed this! I'll look into that tomorrow.

[keks]

Oh I see. So at the moment the log "vanishes" as soon as all peers are offline (which happens every time the process terminates).

Keeping history is non-trivial and afaik @haadcode's JS implementation suffers from the same problem. We've been talking about how to add that but did not come up with a nice solution yet. I really hope we can solve this soon, though.

[haadcode]

In the JS implementation, we save a simple json blob to a local file where the db name is the key and latest hash is the value. The value is saved (cached) on every message that comes through network (https://github.com/haadcode/orbit-db/blob/master/src/OrbitDB.js#L95) and on every local write (https://github.com/haadcode/orbit-db/blob/master/src/OrbitDB.js#L105). On initializing the database, we read that file https://github.com/haadcode/orbit-db/blob/master/src/OrbitDB.js#L75.

Hope this helps!

[jeanlucmongrain]

JS implementation suffers from the same problem

I wanted to compare with JS implementation, couldn't even npm install it and/or it's dependencies... and gave up 😞

On initializing the database, we read that file

ahhh ok, I see! well thanks a lot!

I was looking for a way to get missed events while a node was down. Look like it's not possible a this point!

[haadcode]

I was looking for a way to get missed events while a node was down. Look like it's not possible a this point!

How the log gets "missed" events is that upon re-connecting (to other peer(s)) and receiving a new head from another peer, their log (of missing events) will be fetched and the data gets added to the the database. Ie. there shouldn't be missed events at all. Does that clarify it and answer your question?

I wanted to compare with JS implementation, couldn't even npm install it and/or it's dependencies... and gave up 😞

Sorry to hear this. Would you mind opening a bug report in https://github.com/haadcode/orbit-db and any details as to what fails (npm install log would be very helpful)?

[keks]

In the JS implementation, we save a simple json blob to a local file where the db name is the key and latest hash is the value.

I was thinking about something like this but didn't get around to it yet. I presume you do that in the orbit code, not in orbitdb? I would just add a Import(h hash) function or similar that imports the hash and its references recursively into the orbitdb. That we you can get something like persistence - if your blobstore is persistent. Calling ipfs repo gc will delete your database though. I guess I could also pin the most recent hash. I'll look into that too.