So it uses memory mapped files for all operations? So whether the file is 1 MB or 1 TB, it'll read what it can via streaming? Sorry, not knowledgeable here. Sounds neat though.
1:x uses mmap, so it would return instantly no matter how large a file you give it
0:x uses 1:x and then it splits the content into lines. unfortunately splitting requires copying, so you'd be limited by the amount of ram (let's ignore "swap").
i don't use mmap for writing/amendment yet. i'll be working on it.
the way modern hardware works is like this: every process run by the os has its own view of the (typically 48-bit) address space. a process can request from the os that a part of that address space be "backed" by a certain file. this means that every time the process touches (i.e. tries to read or write) a virgin memory page there (usually page=4k, always aligned), the os will be automatically notified and will make sure to fill it in with actual content from the file, before the process even knows. from that point on, the page will occupy physical ram. if the os is low on memory later, it may decide to free up the page and return it to its previous state.
in effect, data from disk (or any disk-like storage) can be "streamed" while your program uses ordinary array indexing. the word "streaming" though implies reading from start to end in order (which is additionally sped up by prefetch, but that's a different story..); memory mapping is more general - it allows random access.
Assuming you are terminating strings with a NUL, for 0: it may be advantageous to R/W mmap with copy-on-write (i.e. MAP_PRIVATE). Then you can overwrite LF or CRLF with a NUL.
ianal. to the best of my knowledge the project is clean from a legal perspective. ofc, you don't have to be doing something wrong to become a target of copyright trolling.
Yeah, the reason oK was a toy was because John Earnest works for a company that has their own fork of the K3 source, if I remember correctly, which puts him in an area that won't end with one of Kx's harsh lawsuits getting dismissed immediately.
(Not that if he made a high-performance interpreter it'd be a bad thing legally: the reason GNU's programming guidelines are so archaic is partially because everyone implementing GNU early-on had seen the UNIX source code yet they could get around getting sued by writing esoterically.)
