aerial.utils.io
Various supplementary I/O functions and macros. Some new, some
recapitulations of previous extremely useful functionality that is
no longer included in the standard Clojure ecosystem.
->auioReader
(->auioReader fspec rdr)
Positional factory function for class aerial.utils.io.auioReader.
add-rdrwtr
(add-rdrwtr rw)
(add-rdrwtr rw use-auioRdr)
Add the symbol RW to the set of reader and writer operations letio
will look for and handle.
assemble-bindings
(assemble-bindings bv-fds bvec body-fds deps)
Assemble the fd-bindings created from the original binding vector,
the original binding vector bindings, and the fd-bindings created
from the letio body. Basically need to ensure an ordering such that
any values required in the fds-* bindings are placed above those
bindings and above their using forms of fds-* bindings are placed
below those bindings.
close-stream
(close-stream strm)
Close the i/o stream strm.
do-text-file
macro
(do-text-file [in & options] & body)
do-text-to-text
macro
(do-text-to-text [in out] & body)
fd-use
(fd-use)
Returns count of current file descriptors held. Often (typically)
this can be larger (indeed, _much_ larger) than the count actually
being used. This is due to the fact that things like io/read-lines
or equivalent will not close the file unless it is read in
entirety. There are many cases where reading only the first few
lines is what is needed. Further, reading them all just to get a
few is completely loses all advantage of lazy evaluation (not to
mention possible memory blow up)
*** NOTE: (bug!) UNIX only. Uses /proc/<pid>/fd to determine all
descriptors currently held by the process (the JVM here).
force-gc-finalize
(force-gc-finalize)
Tries to force a GC in order to finalize no longer used OS
resources - in particular file descriptors (see fd-use).
*** NOTE: as described by System.gc(), which we use here, this may
or may not actually accomplish the task at hand as it is only a
'suggestion' to the collector to run. If it does run, all no
longer used resoureces should be finalized, i.e., closed and
returned.
get-xform-body-bindings
(get-xform-body-bindings body deps)
Walk sexps in body and for each that satisfiles rdrwtrs? and nss?,
lift the corresponding reader / writer to a binding pair (gensym'd
symbol with r/w value) and replace the r/w subexpr with the binding
var. Returns a pair [fds xbody], where fds is a binding vector of
the new bindings (could be empty) and the xform'd body.
letio
macro
(letio bindings & body)
Like let, but will close all file descriptors opened in the
bindings (and implicitly in body) and will check for certain lazy
readers and will reach into the call and pull out the designated
file, create a binding of a reader for it to a generated sym, and
replace the file in the call with the sym.
map->auioReader
(map->auioReader m__6522__auto__)
Factory function for class aerial.utils.io.auioReader, taking a map of keywords to field values.
open-file
(open-file filespec mode)
Open file filespec (string), accounting for whether it is
gzipped. Mode is either :in or :out. Returns a reader or writer.
NOTE: (bug) file extension of .gz => gzipped, else not
open-streaming-gzip
(open-streaming-gzip fspec io)
Open a gziped file stream on file given by file specification
fspec (a string). The input/output mode is given by io: :in for
input (reading), :out for output (writing)
process-line
(process-line acc line)
read-lines
(read-lines f)
Lazily read lines from text file f by creating a lazying seq over a
buffered reader. Will automatically close the reader after all
lines are read, but will leak the associated file descriptor if the
returned lazy seq is dropped before file is fully consumed. Hence,
this should always be used inside a letio block
read-stream
(read-stream instrm buf)
Read a the next (binary) chunk from input stream instrm into byte
buffer buf. NOTE: this _modifies_ buf!
reduce-file
(reduce-file fname func acc)
Reduce text file denoted by FNAME (filespec/File obj) using
function FUNC per line and reduction seed accumulator ACC. FUNC is
a function of two arguments: first is the current line from file
and second is the current value of ACC.
with-in-reader
macro
(with-in-reader f & body)
Opens a PushbackReader on f, binds it to *in*, and evaluates body.
with-out-append-writer
macro
(with-out-append-writer f & body)
Like with-out-writer but appends to file.
with-out-writer
macro
(with-out-writer f & body)
Opens a writer on f, binds it to *out*, and evalutes body.
Anything printed within body will be written to f.
write-lines
(write-lines f lines)
Writes lines (a collection of strings) to f, separated by newlines.
f is opened with writer, and automatically closed at the end of the
sequence.
write-stream
(write-stream otstrm data)
Write a chunk of output data to output stream otstrm. data is
typically a byte buffer representing whatever data is being written.