Oscaro’s generic I/O tools collection.
Join multiple parts of a path, like os.path.join in Python.
(join-path "foo" "bar") ; => "foo/bar"
(join-path "foo/" "bar") ; => "foo/bar"
(join-path "gs://mybucket" "bar") ; => "gs://mybucket/bar"(basename "/var/log/mysql/") ; => "mysql"
(basename "http://www.google.com/index.html") ; => "index.html"(parent "/var/log/mysql/") ; => "/var/log"
(parent "http://www.google.com/index.html") ; => "http://www.google.com"(splitext "http://www.google.com/index.html") ; => ["http://www.google.com/index" "html"]
(splitext "archive.tar.gz") ; => ["archive.tar" "gz"]return a lazy seq of string from a [protocol://]jsons[.gz] file. warning: the seq must be entirely consumed before the file is closed.
arguments:
- filename: string
returns: an lazy seq of string
return a lazy seq of parsed json objects from a [protocol://]jsons[.gz] file. warning: the seq must be entirely consumed before the file is closed.
arguments:
- filename: string
returns: an lazy seq of parsed objects
example
(doall (map println (read-jsons-file "sample.jsons.gz")))return a lazy seq of parsed edn objects from a [protocol://]edn[.gz] file. warning: the seq must be entirely consumed before the file is closed.
arguments:
- filename: string
returns: an lazy seq of parsed objects
return a lazy seq of parsed csv row as vector from a [protocol://]file.csv[.gz] file. see http://clojure.github.io/data.csv/ for options. warning: the seq must be entirely consumed before the file is closed.
arguments:
- filename: string
- args: options for data.csv
{:separator (default \,) :quote (default \")}
returns: an lazy seq of parsed objects
return a lazy seq of parsed json objects from [protocol://]jsons[.gz] files. warning: the seq must be entirely consumed before every files are closed.
arguments:
- [filenames]
returns: an lazy seq of parsed objects
example
(doall (map println (read-jsons-files ["part1.jsons.gz" "part1.jsons.gz"])))return a lazy seq of parsed json objects from [protocol://]jsons[.gz] files. warning: the seq must be entirely consumed before every files are closed.
arguments:
- [filenames]
returns: an lazy seq of parsed objects
return a seq of filenames beginning with provided path.
arguments:
- path
- [options]
returns: seq of string
examples
(doall (map println (list-files "gs://my-bucket/dir/20160902/animals")))
;-> output:
;gs://my-bucket/dir/20160902/animals-aaaaaaaaaa.jsons.gz
;gs://my-bucket/dir/20160902/animals-aaaaaaaaab.jsons.gz
;gs://my-bucket/dir/20160902/animals-aaaaaaaaac.jsons.gz
;gs://my-bucket/dir/20160902/animals-aaaaaaaaad.jsons.gz
;gs://my-bucket/dir/20160902/animals-aaaaaaaaae.jsons.gz
(doall (map println (list-files "/home/alice/dir/20160902/animals")))
;-> output:
;/home/alice/dir/20160902/animals-aaaaaaaaaa.jsons.gz
;/home/alice/dir/20160902/animals-aaaaaaaaab.jsons.gz
;/home/alice/dir/20160902/animals-aaaaaaaaac.jsons.gz
;/home/alice/dir/20160902/animals-aaaaaaaaad.jsons.gz
;/home/alice/dir/20160902/animals-aaaaaaaaae.jsons.gzreturn a seq of directory under the path directory.
arguments:
- path
- [options]
returns: seq of string
examples
(doall (map println (list-dirs "gs://my-bucket/dir/")))
;gs://my-bucket/dir/20160902/
;gs://my-bucket/dir/20160902/
;gs://my-bucket/dir/20160902/
;gs://my-bucket/dir/20160902/
;gs://my-bucket/dir/20160902/read and parse a configuration file. edn, clj, json, js, yaml, yml supported.
note: if filename is a string, its searched first in resources, then locally
arguments:
- filename (string or io/resource or io/file)
returns: an object
Copy file from source to destination.
arguments
- from
- from-opts
- to
- to-opts
- [copy-opts]: by defaults copy-opts = {buffer-size 1024}
example
Text file:
(copy
"/tmp/windows-file.csv"
{:encoding "windows-1252"}
"gs://my-bucket/dir/uf8-file.csv"
{:encoding "UTF-8" :mime-type "text/csv"}
{:buffer-size 2048})For binary file, you must use an 8-bits encoding:
(def byte-encoding "ISO-8859-1")
(copy
"/tmp/workbook.xlsx"
{:encoding byte-encoding}
"gs://my-bucket/dir/uf8-file.csv"
{:encoding byte-encoding :mime-type "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"}
{:buffer-size 2048})Recursively remove a directory.
(rm-rf "/path/to/my/directory")Create a temporary file and remove it at the end of the body.
(with-tempfile [filename]
(println "There's a file called" filename "."))
(println "The file is now gone.")Create a temporary directory and remove it at the end of the body.
(with-tempdir [dirname]
(println "There's a directory called" dirname "."))
(println "The directory is now gone.")Test if a file exists.
(exists? "https://oscaro.com") ;=> true
(exists? "local-file-that-do-not-exists") ;=> falsereturn a file as map like {:stream clojure.java.io/reader}, with support for [protocol://]file and file.gz.
you need to call (close! file) when you done.
arguments:
- filename
- [options]: by default options = {encoding "UTF-8"}
returns: an map with a :stream key
By default, tools.io supports gzip, bzip2 and framed lz4
compression algorithms and can be extended by implementing a custom protocol
(see sources).
It also supports the following formats if you provide the required dependencies.
An exported clj-kondo with hooks is provided. You can import it using
clj-kondo --lint "$(clojure -Spath)" --copy-configs --skip-lintor, if using lein,
clj-kondo --lint "$(lein classpath)" --copy-configs --skip-lintCopyright © 2016-2024 Oscaro.com
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.