How do I read an existing hypercore on disk?

[rklaehn]

Question

How do I read an existing hypercore on disk?

Your Environment

Software	Version(s)
hypercore	0.10.0
Rustc	rustc 1.37.0 (eae3437df 2019-08-13)
Operating System	macOS 10.12.6 (16G1815)

Question

I was doing some initial evaluation of rust hypercore. One of the things I wanted to try was to read existing hypercores, such as the ones written by mafintosh/hypercore. However, I could not figure out how to do that. Even when passing in the directory of an existing hypercore, the .len() method returns 0.

Context

I want to check compatibility with js hypercore and also compare read performance.

[rklaehn]

Here is what I tried. Forgive the bad rust, just playing around on a Sunday night...

fn read_feed(path: std::path::PathBuf) {
    let mut storage = Storage::new_disk(&path).expect("cannot create storage");
    let mut feed = Feed::with_storage(storage).expect("cannot create feed");
    println!("{} {:x?}", feed.len(), feed.public_key().as_bytes());
    let mut i = 0;
    while let Ok(Some(x)) = feed.get(i) {
        i += 1;
        println!("{} {:?}", i, x);
    }
    println!("{}", i);
}

I get a length of 0, and even if I read the feed I don't get any elements.

[khernyo]

Although I'm not familiar with this repo, from a quick glance, it seems that the on disk data is not read in at all. I guess this is very much a work in progress.

PRs are appreciated! :)

[theorangepotato]

It looks to me like the problem is with Storage::data_offset(), as indicated by this comment. (However, I could be wrong.)

I'm new to this codebase as well, so I'm not quite sure what the problem is, but if anyone else is interested in giving this a try, I believe this is the corresponding Javascript function.

The above may also be a part of the problem, but at the very least it is because while opening the feed it gets the correct keys, but it always assumes it has no data by creating empty data structures and setting its length to 0.