aerial.utils.io

Various supplementary I/O functions and macros. Some new, some
recapitulations of previous extremely useful functionality that is
no longer included in the standard Clojure ecosystem.

->auioReader

(->auioReader fspec rdr)
Positional factory function for class aerial.utils.io.auioReader.

add-rdrwtr

(add-rdrwtr rw)(add-rdrwtr rw use-auioRdr)
Add the symbol RW to the set of reader and writer operations letio
will look for and handle.

assemble-bindings

(assemble-bindings bv-fds bvec body-fds deps)
Assemble the fd-bindings created from the original binding vector,
the original binding vector bindings, and the fd-bindings created
from the letio body. Basically need to ensure an ordering such that
any values required in the fds-* bindings are placed above those
bindings and above their using forms of fds-* bindings are placed
below those bindings.

auio-rdrs

close-stream

(close-stream strm)
Close the i/o stream strm.

do-text-file

macro

(do-text-file [in & options] & body)

do-text-to-text

macro

(do-text-to-text [in out] & body)

fd-use

(fd-use)
Returns count of current file descriptors held.  Often (typically)
this can be larger (indeed, _much_ larger) than the count actually
being used.  This is due to the fact that things like io/read-lines
or equivalent will not close the file unless it is read in
entirety.  There are many cases where reading only the first few
lines is what is needed.  Further, reading them all just to get a
few is completely loses all advantage of lazy evaluation (not to
mention possible memory blow up)

*** NOTE: (bug!) UNIX only.  Uses /proc/<pid>/fd to determine all
    descriptors currently held by the process (the JVM here).

force-gc-finalize

(force-gc-finalize)
Tries to force a GC in order to finalize no longer used OS
resources - in particular file descriptors (see fd-use).

*** NOTE: as described by System.gc(), which we use here, this may
    or may not actually accomplish the task at hand as it is only a
    'suggestion' to the collector to run.  If it does run, all no
    longer used resoureces should be finalized, i.e., closed and
    returned.

get-xform-body-bindings

(get-xform-body-bindings body deps)
Walk sexps in body and for each that satisfiles rdrwtrs? and nss?,
lift the corresponding reader / writer to a binding pair (gensym'd
symbol with r/w value) and replace the r/w subexpr with the binding
var. Returns a pair [fds xbody], where fds is a binding vector of
the new bindings (could be empty) and the xform'd body.

get-xform-rdrwrtr-bindings

(get-xform-rdrwrtr-bindings bvec)
Transform the binding vector bvec by reaching into any value
form (right hand side of pair) which has a reader or writer form (as
from clojure.javas.io) and lifting that form out, creating a binding
for it (with a gensym for the fd bind symbol - left hand side of
binding). Top level reader or writer bindings are left as is. Any
read-lines binding or use in a subform, has its argument lifted into
an explicit reader binding. Returns the xformed binding vector.

letio

macro

(letio bindings & body)
Like let, but will close all file descriptors opened in the
bindings (and implicitly in body) and will check for certain lazy
readers and will reach into the call and pull out the designated
file, create a binding of a reader for it to a generated sym, and
replace the file in the call with the sym.

map->auioReader

(map->auioReader m__6522__auto__)
Factory function for class aerial.utils.io.auioReader, taking a map of keywords to field values.

nss-set

open-file

(open-file filespec mode)
Open file filespec (string), accounting for whether it is
gzipped. Mode is either :in or :out. Returns a reader or writer.

NOTE: (bug) file extension of .gz => gzipped, else not

open-streaming-gzip

(open-streaming-gzip fspec io)
Open a gziped file stream on file given by file specification
fspec (a string). The input/output mode is given by io: :in for
input (reading), :out for output (writing)

process-line

(process-line acc line)

rdrwtrs-set

read-lines

(read-lines f)
Lazily read lines from text file f by creating a lazying seq over a
buffered reader.  Will automatically close the reader after all
lines are read, but will leak the associated file descriptor if the
returned lazy seq is dropped before file is fully consumed.  Hence,
this should always be used inside a letio block

read-stream

(read-stream instrm buf)
Read a the next (binary) chunk from input stream instrm into byte
buffer buf. NOTE: this _modifies_ buf!

reduce-file

(reduce-file fname func acc)
Reduce text file denoted by FNAME (filespec/File obj) using
function FUNC per line and reduction seed accumulator ACC.  FUNC is
a function of two arguments: first is the current line from file
and second is the current value of ACC.

rwfl-syms

with-in-reader

macro

(with-in-reader f & body)
Opens a PushbackReader on f, binds it to *in*, and evaluates body.

with-out-append-writer

macro

(with-out-append-writer f & body)
Like with-out-writer but appends to file.

with-out-writer

macro

(with-out-writer f & body)
Opens a writer on f, binds it to *out*, and evalutes body.
Anything printed within body will be written to f.

write-lines

(write-lines f lines)
Writes lines (a collection of strings) to f, separated by newlines.
f is opened with writer, and automatically closed at the end of the
sequence.

write-stream

(write-stream otstrm data)
Write a chunk of output data to output stream otstrm. data is
typically a byte buffer representing whatever data is being written.