|JTOOLS INTERVAL FILE GENERATOR - STOCKS/INDEXES/OPTIONS
JTools Table of Contents
Project Location: \JTools_IntervalFileGenerator_Stocks
The Interval File Generator for Stocks and Stock Options is very similar to the
File Generator but contains some modifications and special features.
Like the Standard Interval Generator, the data is true interval data. That is
to say that if the timing interval is set to 1 minute then the values generated
are for 1 specific minute in time.
The JTools_IntervalFileGenerator for Stocks/Stock Options allows you to dump
trade and nbbo information in almost any fashion you desire. The system gathers
information from NxCore Trade/Quote messages for interval data. The application
can be run from state files generated just before or at the desired time.
Fields generated and field order are determined by the usage strings (see below
for a complete list of possible fields). It will dump the output to the console
(if specified to) and to a dated text file(s).
Data is output for each day to one file per symbol, or to a data-time series
file containing all symbols (unlike the Standard Interval Generator which
outputs data for all symbols only into date-time series files).
Data is also collected using a different method than the Standard Interval
Generator, in that each symbol has it's own composite structure with X possible
data points (allocated at realtime and dependant on the timeframe requested).
This is possible as the symbol scope (and hence memory usage) is more limited
than the Standard Interval Generator (which can contain all symbols for all
data types, equities, futures, indexes, etc).
Another feature not possible with the standard interval generator is the
ability to correct bar data based on cancellations and trade corrections by
capturing Category 25 Bar
The JTools Interval Generator for Stocks/Stock Options also combines the
functionality of the JTools Batch
Processor to make processing data which spans days, months or years
Interval data timestamps will mark the beginning of the interval. For instance,
if 5 minute interval data is chosen, an interval at 9:30 will contain data from
9:30:00 through 9:34:59. The Open price reflects the first price recorded for
the specific interval. The last price will represent the last price recorded
for the specific interval. The high and low price reflect the highest price and
lowest price during the specific interval. Volume will reflect the total volume
accumulated during the specific interval.
Request Data Parameters:
- Read Symbols From File - If checked, symbols to output will be read
from the text file specified in Sym File.
- Sym File - You may read the symbols you are interest in generating
data for from a text file. The Sym File designates the file. You may type it in
or Browse for the specific file. Read Symbols From File must be
A sample file, "Symbols.txt" is provided in the project folder.
- Stock Symbol - The symbol(s) to be output (if Read Symbols
From File is disabled).
The symbol(s) may be entered in one of the following manners:
- A single symbol, such as " eAAPL "
- Multiple, comma delineated symbols, such as " eAAPL,iSPX,eGOOG,iDJI
- A wildcard symbol, such as " eA* ". Note that if a wildcard is
entered it must be the only entry.
- Generate Options Data - If checked, options for the underlying
stocks and/or indexes will also be generated.
- Listed Exg Filter - "0"=Include all exchanges. If only
specific exchanges are desired, enter those exchange here, comma separated. IE
for NYSE and NQEX - 3,12
Time and Interval Period Parameters:
- Start Time (HHMM) - Time that the application will start
collecting/generating interval data. Example - "09:30".
- Stop Time (HHMM) - Time that the application will stop
collecting/generating interval data. Example - "16:00".
- Bar Interval (1-60) - Amount of minutes each interval represents.
Data Field Parameters:
- Data Usage/Output String - Possible values:
(s) Symbol (e) Listed Exg (d) Date (t) Time (1) Open (2) High (3) Low (4) Last
(5) NetChg (6) Total Volume (7) Tick Volume (a) Best Ask (b) Best Bid (8) Best
Bid Size (9) Best Ask Size (v) VWAP (O) Open Interest (U) Underlying Symbol (/)
The field order of the output data will match the field order of the Data Usage
- Print Headers - Print headers in output file if checked.
- Exg Codes as Text - If checked, exchange codes are output as text,
otherwise they are output as numeric.
- Process Empties - If checked issues with no data for the interval
will be included and zeroed. If unchecked, only intervals where trading data
was recorded will be output.
- Carry Through OHLC on Empty- If checked, a bar that has no trade
data will carry the previous bar values for the Open,High,Low,Last values.
- Carry Through Bid Ask on Empty - If checked, a bar that has no new
quote data will carry the previous bar values for the Best Bid, Best Ask, Best
Bid Size and Best Ask Size.
- Output Symbol Prefix - If checked symbol prefixes will be output
with the symbol.
- Decimal precision - ("0" - "5", "-1"
for auto based on price type).
- Price Filter Level - NxCore halt built in analysis on every trade
that can help determine possible erroneous pricing (which may cause spikes when
charting the data). One such item is the Filter Level field within the
NxCTAnalysis structure. You may see the full documentation for this
For example, if you wish to for eliminate all trades with a filter level of 6
or higher, set the Price Filter Level to 6. Note that the volume for these
trades will still be included in the total interval volume, but the pricing
will be ignored.
- Process Saturday - If checked files will be generated on Saturday.
- Process Sunday - If checked files will be generated on Sunday.
- Odd Lots Can Set Price - Technically, Odd Lot trades are not
eligible to set the last sale price (although volume is considered in official
total volume) and therefore do not set any pricing fields in the interval
However, in some equities it may be advantageous to include Odd Lot pricing,
esp. in illiquid stocks where trading is thin and odd lots may in fact be the
only trades in a specific interval.
Enable this option if you wish to have odd lots included in the interval
- Odd Lots Can Set Volume - Technically, volume from Odd Lot trades
are included in the current official total volume. (although are not eligible
to set the last sale price).
Some of our users however desire to have Odd Lot volume excluded from the
interval volumes generated.
Disable this option if you wish to have odd lot volume excluded from interval
Output File Parameters:
- Write On Interval - If checked, data files will be generated when
each time interval is passed. No corrections are available for data generated
in this manner. May be used in conjunction with Write On
- Write On End-Of-Tape (or Stop Time) - If checked, data files will be
generated either a) when the specified Stop Time has been reached or b) at the
end of the tape (see below). May be used in conjunction with Write On
- Continue Until Complete Msg - If checked and Write On End Of
Tape is also specified, data will be written when the tape completes as
opposed to the specified Stop Time. This allows any corrections to be processed
as corrections do not occur until after the actual trade data, sometimes much
- Symbol Based Files - If checked, files will be named and output
according to the stock symbol. If unchecked, all symbols will be named and
output to a single date-time based file.
- Append - One File Per Symbol - If checked, data will be written to
one file per symbol, continuously. Symbol Based Files and
Write On End-Of-Tape options must also be enabled (checked on).
Note: If appending to one file per symbol, the files are never overwritten with
new data, but the existing data is appended to. Therefore, should you change
parameters in the application and wish to regenerate the data from scratch, you
must first Delete the existing files. Use the Delete Files button to accomplish
this, or manually delete them through Windows Explorer.
- Default File Extension - The default file extension to use for
- Field Separator - A single character used to separate data fields
within the files.
- Data Folder - Folder data files are created in.
If left blank, the output files will be generated in the sub-folder
JTools_OutData\IntervalData\Stocks. The sub-folder will be located in the same
folder that the application runs from.
- Delete Files - Deletes all files in the Data Folder with filenames
ending with ".IFGSIO.<Default File Extension>" .
You will be asked when pressing this button if you are sure, before the
Batch Mode Parameters:
- Run in Batch Mode - If checked, batch parameters become enabled and
allow the application to process files in a "Batch Mode". If
unchecked, individual files or realtime mode may be chosen in the standard
- Start Date- Date of the first historical file to be processed.
- Stop Date- Date of the last historical file to be processed.
- Tape Set- The NxCore 2 or 3 letter tape set code.
- Tape Ext- The NxCore tape extension (nxc, nx2 or nx3).
- Tape Path- Path to the historical tapes.
- Use Year Subfolders - Many users will organize their historical
tapes by year (such as Data\2016, Data\2015, etc). If 'Use Year Subfolders' is
enabled then the application will search in the Tape Path folder + the year
sub-folder for the specific NxCore historical tape file.
- Continue on Errors - If checked and an error occurs (such as a
missing file in the sequence) processing will continue with the next file. If
unchecked processing will stop on the first error encountered.
- Run in Reverse Direction - When selected, Batch processing will
occur from the stop date to the start date. If not selected batch processing
occurs from the start date to the stop date.
- Output Data to Activity Monitor - Output of the actual interval data
will be displayed in the activity monitor if checked. While this can be useful
to determine if data is actually being generated, it can severely impede
performance of the application.
- Output File Status to Activity Monitor - Output of the file write
status will be displayed in the activity monitor if checked. Note any file
errors will be reported regardless of this setting.
Project Location: \JTools_IntervalFileGenerator_Stocks
JTools Table of Contents