FSDB - File System Data Base¶
Contents:
- Introduction
- Installation
- Example Usage
- A larger example
- PyFSDB Command Line Tools
- Data filtering and modification tools
- pdbrow - select a subset of rows based on a filter
- pdbroweval - alter rows based on python expressions or code
- pdbensure - ensure certain columns are present in the data
- pdbaugment - join rows from one FSDB files into another
- pdbfgrep - join rows from one FSDB files into another
- pdbnormalize - normalize a bunch of columns
- pdbcdf - find all unique values of a key column
- pdbdatetoepoch - translate a date-string based column to unix epochs
- pdbepochtodate - translate a unix epoch column to a date-string column
- pdbkeyedsort - find all unique values of a key column
- pdbsum - sum columns together
- pdbzerofill - fills a columns with zeros (or other value) when blank
- Data conversion tools
- pdb2to1 - strip typing information from the FSDB header
- pdbaddtypes - strip typing information from the FSDB header
- pdbformat - create formatted text per row in an FSDB file
- pdbjinja - process an FSDB file with a jinja template
- pdb2tex - create a latex table using the data in a FSDB file
- pdb2sql - uploads an FSDB file into a database
- pdbsplitter - split an FSDB file into multiple files
- pdbfullpivot - translate a date-string based column to unix epochs
- pdbreescape - regexp escape strings from a column
- Data analysis tools
- Data filtering and modification tools
- Perl FSDB Tools
- cgi_to_db - convert stored CGI files (from CGI.pm) to fsdb
- combined_log_format_to_db - convert Apache Combined Log Format to Fsdb
- csv_to_db - convert comma-separated-value data into fsdb
- dbcolcopylast - create new columns that are copies of prior columns
- dbcolcreate - create new columns
- dbcoldefine - define the columns of a plain text file to make it an Fsdb
- dbcolhisto - compute a histogram over a column of Fsdb data
- dbcolize - DEPRECATED, now use dbfilealter
- dbcolmerge - merge multiple columns into one
- dbcolmovingstats - compute moving statistics over a window of a column
- dbcolneaten - pretty-print columns of Fsdb data (assuming a monospaced
- dbcolpercentile - compute percentiles or ranks for an existing numeric
- dbcolrename - change the names of columns in a fsdb schema
- dbcol - select columns from an Fsdb file
- dbcolscorrelate - find the coefficient of correlation over columns
- dbcolsplittocols - split an existing column into multiple new columns
- dbcolsplittorows - split an existing column into multiple new rows
- dbcolsregression - compute linear regression between two columns
- dbcolstatscores - compute z-scores or t-scores for each value in a
- dbcolstats - compute statistics on a fsdb column
- dbcoltype - define (or redefine) types for columns of an Fsdb file
- dbfilealter - alter the format of an Fsdb file, changing the row/column
- dbfilecat - concatenate two files with identical schema
- dbfilediff - compare two fsdb tables
- dbfilepivot - pivot a table, converting multiple rows into single wide
- dbfilestripcomments - remove comments from a fsdb file
- dbfilevalidate - insure the source input is a well-formed Fsdb file
- dbformmail - write a shell script that will send e-mail to many people
- dbjoin - join two tables on common columns
- dblistize - DEPRECATED, now use dbfilealter
- dbmapreduce - reduce all input rows with the same key
- dbmerge2 - merge exactly two inputs in sorted order based on the the
- dbmerge - merge all inputs in sorted order based on the the specified
- dbmultistats - run dbcolstats over each group of inputs identified by
- dbrecolize - DEPRECATED, now use dbfilealter
- dbrowaccumulate - compute a running sum of a column
- dbrowcount - count the number of rows in an Fsdb stream
- dbrowdiff - compute row-by-row differences of some column
- dbrowenumerate - enumerate rows, starting from zero
- dbroweval - evaluate code for each row of a fsdb file
- dbrow - select rows from an Fsdb file based on arbitrary conditions
- dbrowuniq - eliminate adjacent rows with duplicate fields, maybe
- dbrvstatdiff - evaluate statistical differences between two random
- dbsort - sort rows based on the the specified columns
- dbstats - DEPRICATED, now use dbcolstats
- db_to_csv - convert fsdb to the comma-separated-value file-format
- db_to_html_table - convert db to an HTML table
- html_table_to_db - convert HTML tables into fsdb
- kitrace_to_db - convert kitrace output to Fsdb format
- ns_to_db - convert one of ns’s output format to jdb
- sqlselect_to_db - convert MySQL or MariaDB selected tables to fsdb
- tabdelim_to_db - convert tab-delimited data into fsdb
- tcpdump_to_db - convert tcpdump textual output to fsdb
- xml_to_db - convert a subset of XML into fsdb
- yaml_to_db - convert a subset of YAML into fsdb
- FSDB Class Description
FsdbFsdb.append()Fsdb.close()Fsdb.column_namesFsdb.comment()Fsdb.commentsFsdb.convert_separator_token()Fsdb.convertersFsdb.create_header_line()Fsdb.extend()Fsdb.file_handleFsdb.filter()Fsdb.foreach()Fsdb.get_all()Fsdb.get_column_name()Fsdb.get_column_number()Fsdb.get_column_numbers()Fsdb.get_pandas()Fsdb.guess_converters()Fsdb.header_lineFsdb.headersFsdb.maybe_open_filehandle()Fsdb.next_as_array()Fsdb.next_as_dict()Fsdb.out_column_namesFsdb.out_command_lineFsdb.out_fileFsdb.out_file_handleFsdb.out_header_lineFsdb.out_separatorFsdb.out_separator_tokenFsdb.parse_commands()Fsdb.parse_separator()Fsdb.put_all()Fsdb.put_pandas()Fsdb.read_commands_ahead()Fsdb.read_header()Fsdb.row_as_string()Fsdb.separatorFsdb.separator_tokenFsdb.set_iterator_function()