Discussion:
Stock exchange-related data
(too old to reply)
A.D. Fundum
2011-12-22 17:16:50 UTC
Permalink
Just checking, albeit I'm quite busy at the moment:

Is there a demand for a generic stock exchange-related data format?

I've a collection of small apps, which generates unique data, but I'v
never shared those apps because it assumes my data model. And I don't
want to support all possibe data formats used by unknown data vandors
in unclear countries. A made-up example of an app is a top10-gainers
generator, using a HTML file for its output. Or a list of stocks where
the MA-10 crossed the MA-200,

Random example: on a scale of 1 to 10, I'ld rate the data quality of
the NYSE itself with an 8. Their data often contains errors (-1), and
not all errors are repairable (again -1). But because I use my onwn
data format, not just for the quotes, it has no use to publish some
DIFF file with corrected errors. I also have to check it each day,
bevause nobody else has a DIFF file. Or a DIFF file processor. Or an
app for corporate actions-related data maintenance.

If there's a merger between A and B, I'll continue with AB and delete
all old data of A and B. Lots of data, lots of choices. Apps may even
be specific for a country, e.g. an app scanning for an action without
(other) buyers.

Please note the question is related to stock exchange-related data
formats/models. Saying "yes" doesn't mean you'll receive the latest
2012 estimated EPS data of the French stock exchange for free. Saying
may mean there'll be a converter between your format and the generic
format, so you can use all apps using the generic format. Apps may not
be free (albeit I'm not selling apps), data may not be free, services
may not be free, and an silly app may just answer a stupid newsgroup
idea ("is it a good idea to buy stocks that gained during at least 10
days"?) using a textmode interface.


--
Will Honea
2011-12-22 18:55:02 UTC
Permalink
Post by A.D. Fundum
Is there a demand for a generic stock exchange-related data format?
I've a collection of small apps, which generates unique data, but I'v
never shared those apps because it assumes my data model. And I don't
want to support all possibe data formats used by unknown data vandors
in unclear countries. A made-up example of an app is a top10-gainers
generator, using a HTML file for its output. Or a list of stocks where
the MA-10 crossed the MA-200,
Random example: on a scale of 1 to 10, I'ld rate the data quality of
the NYSE itself with an 8. Their data often contains errors (-1), and
not all errors are repairable (again -1). But because I use my onwn
data format, not just for the quotes, it has no use to publish some
DIFF file with corrected errors. I also have to check it each day,
bevause nobody else has a DIFF file. Or a DIFF file processor. Or an
app for corporate actions-related data maintenance.
If there's a merger between A and B, I'll continue with AB and delete
all old data of A and B. Lots of data, lots of choices. Apps may even
be specific for a country, e.g. an app scanning for an action without
(other) buyers.
Please note the question is related to stock exchange-related data
formats/models. Saying "yes" doesn't mean you'll receive the latest
2012 estimated EPS data of the French stock exchange for free. Saying
may mean there'll be a converter between your format and the generic
format, so you can use all apps using the generic format. Apps may not
be free (albeit I'm not selling apps), data may not be free, services
may not be free, and an silly app may just answer a stupid newsgroup
idea ("is it a good idea to buy stocks that gained during at least 10
days"?) using a textmode interface.
I've lately begun reviving a particular application I did several years back
to track my own investments. I pretty much switched to Linux a while back
so the code is still in an OS/2 archive but I'm slowly getting around to
porting the GUI to what should be cross-platform (qt-based). Essentially, I
use Yahoo for quotes and history to keep track of current value, cost basis,
etc. so that I have some clue as to tax implications and such for my IRA and
401k accounts as well as other cash and investment accounts. It's about the
only app I still regularly boot OS/2 to run.

I never distributed the app - like you, I'm not big on playing help desk and
dealing with potential SEC liabilities but it should be possible to gather
enough of a group to get a bunch of this stuff integrated into a useful
package. I started on mine when I found essentially nothing out there for
retirement/estate planning when I entered into that phase of life. Lots of
sales gimmicks but nothing even approaching comprehessive data oriented
capability.
--
Will Honea
A.D. Fundum
2011-12-23 03:01:41 UTC
Permalink
Post by Will Honea
Post by A.D. Fundum
Is there a demand for a generic stock exchange-related data
format?
Post by Will Honea
I've lately begun reviving a particular application I did several years back
to track my own investments. I'm slowly getting around to porting
the GUI
Post by Will Honea
to what should be cross-platform (qt-based).
Same here, sort of. A Rexx prototype most likely was a reply to an
interesting idea or challenging question. If it was good enough, parts
may have been rewritten in C and HTML output often is the UI (in the
WPS environment).

Some of those apps can be used by other people, and some apps don't
have to be embedded in a single app/environment at all. A lot of my
apps produce data specific for a stock exchange or country, or produce
unique data, or verify data quality (today 2 verified errors in NYSE's
volume data, and yesterday 2 verified errors too).
Post by Will Honea
Essentially, I use Yahoo for quotes and history to keep track of
current value,
Post by Will Honea
cost basis, etc.
I used to use Yahoo to complete missing data, but popular Google
("NYSE:MMM") and Yahoo ("MMM") symbols are examples of data that can
be shared. I may once need it to obtain the Yahoo yields you're using,
and if it ever changes to "3M" you may not have to find out the hard
way what happened. And if those sites ever change their layout, other
people already may have re-invented that wheel.
Post by Will Honea
I never distributed the app - like you, I'm not big on playing help
desk
Post by Will Honea
and dealing with potential SEC liabilities but it should be
possible to
Post by Will Honea
gather enough of a group to get a bunch of this stuff integrated
into a
Post by Will Honea
useful package.
Right, hence the original question. A shared data format isn't that
hard. Volume remains volume. A shared data format can be just a few
file format definitions or it can be hidden under the hood. A a matter
of fact, I already had a simple converter for Yahoo's data (FTTOMH it
required changing the symbol, changing the order of downloaded lines,
and the date had to be converted).

Even if I never use your apps, for whatever reason, I still don't mind
adding Yahoo's symbols to my stock definition file(s). The few
American stocks in my list may even help you, if Boeing ever chanches
its name, I may change the Yahoo symbol as well, and tell everybody
what has happened, possibly with a DIFF file and links to
resources/evidence. Or I could verify an investment strategy with data
you don't have. Besides the quotes, other "properties" are indeed
Yahoo's symbols, or personal properties (country, preferred data
quality, data sources, ...).
Post by Will Honea
I started on mine when I found essentially nothing out there for
retirement/estate planning when I entered into that phase of life.
Lots of sales gimmicks but nothing even approaching
comprehessive data oriented capability.
Same with websites, AFAICT. Even Yahoo isn't as good as it was, e.g.
no EPS estimates or dividend-related data w.r.t. Shell at its main
market (RDSA.AS, but that's not that important now), at least not at
the moment. Is "single trick pony" a right phrase? Or perhaps DIY? I
don't use so-called technical analysis at all, but I cannot recall a
single app or website telling you when all of the alarms bells are
ringing. Apparently everybody is still looking at their own
graph/spreadsheet, while a computer can be quite good in removing most
of the graphs you wouldn't care about. And if there's some portfolio
manager. It may not be good enough, and/or it uses its own data format
so you still cannot use your data.

I hestitate mentioning a "gap in the market", for one because there
may not be a market. But it's quite clear there's a gap, and my own
data format (mostly based on Metastock) is preventing me from sharing
any app. I'm not dreaming about some Open Sourced Data Format as such,
the data really is the main problem. Not to mention ASCII data stored
per year, so the files aren't becoming too large for e.g. Rexx or
manual editing. But there aren't that many "properties" of
data/companies/people.

One of your "properties" may be that you care about e.g. the Belgian
or Canadian stock market, or at least a part of that market (yet
another "property").

In a nutshell, for people not having written such apps: if you want to
use my statistically-best-part-of-a-day-to-buy-stocks-app with your
Belgian stock exchange data of Yahoo, you'll now have to, AFAIK:

- convert it to my data layout (number of fields, order of fields)
- change the format of the date
- reverse the order of the data
- split the data in a file per year
- store the data in a specific directory
- add/rename 3 indexes to use my fixed index names

Albeit I'm not from Belgium, once I've gathered the data required:

- translate the output (optional)

Your milage will vary (different timezone), but it looks like it's a
good idea to buy stocks tuesday afternoon and to sell 'em wednesday
morning (open wednesday-close tuesday). There's no documentation and
there are no clear definitions because I'm the only user, and there's
no explaination of the results (most likely it's related to Wall
Street and the USA). IIRC this was actually inspired by an internet
what's-the-best-day-of-the-week-to-...-question. In order to use this
silly app: write your own (DIY), or use a standard data format I'm
using too. I hardly have to modify my app (hard-coded index names) in
order to apply it to your local data. Main showstoppers if you want to
avoid DIY and re-inventing wheels: what's your data format, and what's
your local index. It may be nice-to-have, this pony (BEL20: main
index, BELM: midcaps, BELS: smallcaps), so long as it isn't a
stand-alone pony:


BEL20 high-low monday : 37.57
BEL20 high-low tuesday : 36.97
BEL20 high-low wednesday : 39.41
BEL20 high-low thursday : 39.84
BEL20 high-low friday : 37.84
BELM high-low monday : 34.12
BELM high-low tuesday : 35.72
BELM high-low wednesday : 32.01
BELM high-low thursday : 33.15
BELM high-low friday : 31.83
BELS high-low monday : 53.65
BELS high-low tuesday : 54.46
BELS high-low wednesday : 47.32
BELS high-low thursday : 47.69
BELS high-low friday : 50.10
BEL20 close-open monday : -6.47
BEL20 close-open tuesday : 0.67
BEL20 close-open wednesday: -0.79
BEL20 close-open thursday : -3.39
BEL20 close-open friday : -1.84
BELM close-open monday : -8.77
BELM close-open tuesday : 0.04
BELM close-open wednesday: -3.09
BELM close-open thursday : -8.35
BELM close-open friday : -2.04
BELS close-open monday : -14.72
BELS close-open tuesday : -5.08
BELS close-open wednesday: -0.38
BELS close-open thursday : -11.88
BELS close-open friday : 2.72
BEL20 higher/lower/unchanged monday : 25/26/0
BEL20 higher/lower/unchanged tuesday : 25/27/0
BEL20 higher/lower/unchanged wednesday: 24/28/0
BEL20 higher/lower/unchanged thursday : 25/28/0
BEL20 higher/lower/unchanged friday : 23/28/0
BELM higher/lower/unchanged monday : 22/29/0
BELM higher/lower/unchanged tuesday : 29/23/0
BELM higher/lower/unchanged wednesday: 23/29/0
BELM higher/lower/unchanged thursday : 20/33/0
BELM higher/lower/unchanged friday : 24/27/0
BELS higher/lower/unchanged monday : 22/29/0
BELS higher/lower/unchanged tuesday : 23/29/0
BELS higher/lower/unchanged wednesday: 25/27/0
BELS higher/lower/unchanged thursday : 22/31/0
BELS higher/lower/unchanged friday : 26/25/0
BEL20 open monday -close friday : -4.49
BEL20 open tuesday -close monday : -0.80
BEL20 open wednesday-close tuesday : 3.02
BEL20 open thursday -close wednesday: 0.64
BEL20 open friday -close thursday : 1.96
BELM open monday -close friday : -1.17
BELM open tuesday -close monday : 0.99
BELM open wednesday-close tuesday : 4.37
BELM open thursday -close wednesday: 1.87
BELM open friday -close thursday : 2.76
BELS open monday -close friday : 0.95
BELS open tuesday -close monday : 1.70
BELS open wednesday-close tuesday : 7.97
BELS open thursday -close wednesday: 0.86
BELS open friday -close thursday : 0.41


--
Will Honea
2011-12-23 05:50:07 UTC
Permalink
Post by A.D. Fundum
- translate the output (optional)
Revealing a bit of my former life, I tend to put everything into a formal
database. Personally, I use DB2 as I have 20-odd years exposure to it but
SQLITE would work nearly as well. My philosophy is that you NEVER store a
numerical value more than once. If it isn't in the right format/collection,
link and index it. If you have multiple exchanges, add a column for each
token you need or - if the row count varies drastically - use subtables. I
fought database design for a couple of multi-national corporations for a
long time and the one thing that stuck was that any time spent in design of
the data repository is very quickly recovered as soon as you start writing
queries.

I have a close friend who makes a very good living publishing market
forecasts based on chaos theory. His inputs include the usual stuff plus
moon phase, weather conditions, planetary alignment and a few REALLY odd
considerations. I kid him about figuring out how to get the chicken blood
included but I envy his results. His net worth is several times my own ;-)

I'm not much of a trader myself but I do like to track performance of what I
hold.
--
Will Honea
A.D. Fundum
2011-12-24 02:20:12 UTC
Permalink
Post by Will Honea
Post by A.D. Fundum
Albeit I'm not from Belgium, once I've gathered the data
- translate the output (optional)
Revealing a bit of my former life, I tend to put everything into a
formal database.
My data model is also database'ish, but I'm using text files because
I'm not a programmer, I used Rexx first, and other people may share or
understand my data without having to have a certain database. It also
avoids e.g. SQL errors (blame my limited SQL skills).The main
difference is a stock quotes file per year. My original files were
organized that way, and the files are manageble due to the smaller
size. The first two lines of such a sorted file/table, albeit I don't
actually use the header (an API could use it, but my own file format
is fixed):

----- 8X -----
<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OI>
3WPW,D,20110103,0.20,0.20,0.20,0.20,0
----- 8X -----

And a sorted file/table with data per listed stock, e.g.

----- 8X -----
3WPW 3W Power Hold. Warr.GG00B39QCZ84 N
----- 8X ----

That's just the symbol/ticker, the name choosen by me, and the
ISIN-code required to download NYSE's data. The N (of "profit
expected: No") was added later, just like a missing character
indicating in which market segment this stock can be found.
Post by Will Honea
My philosophy is that you NEVER store a numerical
value more than once.
I do, but you can consider that such data represents a "temporary
table with calculated fields". The power of an EXE is used to crunch
the number and create a file (not relevant for a data model), and a
Rexx app may format the data using this raw ouput file. I keep most of
those files intentionally, for research purposes. Even if a delisting
means that I sometimes have to recreate my data files.
Post by Will Honea
If you have multiple exchanges, add a column for each
token you need or - if the row count varies drastically - use
subtables.
Same here. I have a main directory, and a subdirectory for secondairy
exchanges. Symbols may be re-used, and in my case the same stock is
listed at multiple stock exchanges with exactly the same data (data
stored twice). I could use a single directory, but it just grew this
way.
Post by Will Honea
I fought database design for a couple of multi-national
corporations
Post by Will Honea
for a long time and the one thing that stuck was that any time
spent
Post by Will Honea
in design of the data repository is very quickly recovered as soon
as you start writing queries.
What I was rouglhy thinking about maybe involves 4 tables, without
wanting to call any shot: stock quote data (quotes), stock data (full
name, EPS, qualifies for a 401k, Yahoo symbol, Google symbol, and so
on). A third table with pesonal preferences (codepage, if A and B
merge do you want to keep A's and B's old data, do you want an email
if somebody noticed a possible problem with your data, stocks you
hate, and so on), a fourth table with currencies and exchange rates
(if you care about that, according to table 3) and a table with global
preferences (main index, stock exchanges you care about, fixed parts
of download URLs, and so oon). That's 4 or 5, albeit a currency can be
seen as a stock. And number 5 or 6: your portfolio, virtual or real.
By the way, I use a main index to determine trading days (e.g. the
last day of the year) per stock exchange, and a special index rule is
that their volume is 0, despite there's a different high and low
(details, details).

Regardless of the use of a database or text files, and so on: indeed
you don't have to maintain all data, and your apps just need the data
they require. For most people my data may be (virtually) shrinked to
e.g.:

<TICKER>,,<DTYYYYMMDD>,,,,<CLOSE>,
3WPW,,20110103,0.20,

And the author of an app just needs the specifications for queries
(being a certified SQL idiot: select close from 2011.XET where
ticker='3WP' and date='20110103'). You don't need the other data, as
long as you don't have an app using it.
Post by Will Honea
I have a close friend who makes a very good living publishing
market forecasts based on chaos theory. His inputs include
the usual stuff plus moon phase, weather conditions, planetary
alignment and a few REALLY odd considerations. I kid him
about figuring out how to get the chicken blood included but I
envy his results. His net worth is several times my own ;-)
My data is inspired by such a model. It's so good scientists don't
believe it (in their words: the results are unlikely). A "blind" (sp?)
hit ratio of over 98%, which means that over 98% of predicted
outperformances turn out to be extremely right. The accuracy
approaches 100% if I'm allowed to remove known errors from the list.
Hence the "N" for EPS, instead of a more common value. But of course
I'm able to convert an EPS of "-0.10" to a N. In an offtopic nutshell:
albeit the weather is a chaotic system, chances are a temperature of
100F is related to your summer. That's a logical TRUE, and I'm
combining a few of those veriafyable remarks.
Post by Will Honea
I'm not much of a trader myself
Nor am I, but my apps should be able to tell you which stocks are
popular by trader for a good reason. Like I said, it may have been an
answer to an internet question. Or I want to have an overview of a
foreign stock market, including signs of pump 'n dump activities.
Post by Will Honea
but I do like to track performance of what I hold.
And I like to add that I also was thinking about an earlier remark of
you, which may come down to a single container app, containing a lot
of small apps, perfomance tracking, silly stats, graphs, whatever.

For a data model we'll need more people than 2, as such it isn't
limited to an OS, and I'ld like to see some global representatives
(main markets and countries will do).

Regarding a database: all I need is an interface to convert (or copy)
my data to such a generic database format, so I'm able to execute a
"select close from 2011.XET where ticker='3WP' and date='20110103'". I
think free is important (not counting apps, your friends app may be
worth an afwul lot of money), and I'll strongly advocate data quality
(for example, this week 5 errors in NYSE data: 4 wrong volumes, 1
stock without a symbol).

Apps don't have to be cool/fast, apps just are using the data. I'ld
like to avoid an open source data model, because then "everybody" can
add fields to a file/table. If there's an EPS and a quote, somebody
will add the calculated P/E field. Regarding tools: nearly 2 years of
data for 2 stock exchanges equals about 5 MB of data, unZIPped (I only
ZIP data older than year-1).


--
Will Honea
2011-12-24 06:42:31 UTC
Permalink
Post by A.D. Fundum
And I like to add that I also was thinking about an earlier remark of
you, which may come down to a single container app, containing a lot
of small apps, perfomance tracking, silly stats, graphs, whatever.
That's always a good idea even if the visible app is just a menu that
invokes the different functions. Gives you a place to park online
documentation and instructions for use as well as keeping thing organized.
Post by A.D. Fundum
For a data model we'll need more people than 2, as such it isn't
limited to an OS, and I'ld like to see some global representatives
(main markets and countries will do).
Not necessarily true. Just reading through you comments I was doodling and
came up with a pretty decent representation of what you described.
Post by A.D. Fundum
Regarding a database: all I need is an interface to convert (or copy)
my data to such a generic database format, so I'm able to execute a
"select close from 2011.XET where ticker='3WP' and date='20110103'". I
think free is important (not counting apps, your friends app may be
worth an afwul lot of money), and I'll strongly advocate data quality
(for example, this week 5 errors in NYSE data: 4 wrong volumes, 1
stock without a symbol).
Don't re-invent the wheel! The simplest way to transport data between about
any data storage programs is the CSV (Comma Separated Variables). That way,
users can define their own storage mechanism. It also has the advantage of
being directly loadable/exportable by virtually any spread sheet app and is
human readable to boot. That makes writing a script to filter your existing
data into a single storage mechanism simple and easily done in you favorite
scripting language - from a DOS command program to BASIC to something really
fancy in C or Java. You likely have about everything pretty much in hand
already just be changing the output format slightly. I'm not real current
on the OS/2 apps out there right now but the last time I looked I think I
remember SQLLITE being available for OS/2.

I would disregard the user's preference for a storage app and concentrate on
a simple definition of a good app agnostic description of the major blocks
(tables or spread sheet layout) just so the end user knew the order and
grouping of columns for each entity in the database plus the necessary
relationships between them and let it go. Publish a detailed schema for
what you use and let it go at that. I can do a whole lot of error checking
and data validation with most database programs (even FoxPro could do that)
but that's a user enhancement beyond the most basic functions of lcating and
aquiring the data in the first place. I have access to some NYSE feeds and
they boil down to the same final data. Account access and such are highly
specific but once you get the data streaming it's all pretty generic.

The reply-to email in my headers is good - we might want to move to email to
continue the discussion.
--
Will Honea
A.D. Fundum
2011-12-29 23:10:35 UTC
Permalink
Post by Will Honea
Post by A.D. Fundum
I like to add that I also was thinking about an earlier remark
of you, which may come down to a single container app,
containing a lot of small apps, perfomance tracking, silly
stats, graphs, whatever.
That's always a good idea even if the visible app is just a
menu that invokes the different functions. Gives you a place
to park online documentation and instructions for use as well
as keeping thing organized.
Exactly. The usage frequency of most of my "apps" was so low that I've
deleted them after the original discussion was finished, so I'm not
claiming the <Alt-S> short-cut for my Silly Stocks Stats. On the other
hand, it won't become a "fast one touch trading app". It's just about
a shared data format.

Perhaps a more logical next step is related to sharing data. If I know
you often provide reliable data regarding Mexican corporate events or
changed US Yahoo! symbols, I may provide you (anonymously) data
regarding a foreign stock exchange in order to check/"backtest" your
sun-phase based theories.
Post by Will Honea
Post by A.D. Fundum
For a data model we'll need more people than 2, as such it
isn't limited to an OS, and I'ld like to see some global
representatives (main markets and countries will do).
Not necessarily true. Just reading through you comments I was
doodling and came up with a pretty decent representation of
what you described.
Yes, in reality it's quite easy to add fields to a table. I think I
don't prefer to use Yahoo! or Google as some backup data source (due
to rounding of the provided Historical Prices), but e.g. adding the
Yahoo! symbols to my data is no problem. It may be useful in case of
some error (normally I use faulty data provided by the primary
source,NYSE, itself). If anything, I can answer a question about a
hard-to-guess Yahoo! symbol.

I guess I was thinking about possible international issues and DBCS
company names. We may assume that 20 characters is more than enough to
hold "Bangkok Intl. Hotel", but I'm afraid you'll need more than 10
bytes to store "Krung Thep Mahanakhon Amon Rattanakosin Mahinthara
Ayuthaya Mahadilok Phop Noppharat Ratchathani Burirom Udomratchaniwet
Mahasathan Amon Piman Awatan Sathit Sakkathattiya Witsanukam Prasit
Intl. Hotel".
Post by Will Honea
Don't re-invent the wheel!
No worries, I'm using an existing file format for my stock quote data,
with headers (no need to use those if you use the standard, but it
allows you to use MMDDYYYY instead of YYYYMMDD). I used a fixed field
length format for my company-related data, but that had more to do
with Rexx' PARSE and readability. No standard there, and I'm using 5
fields max (not counting one-off apps, which may e.g. use Yahoo!'s EPS
data).

I also adobted an implied storage per year, and older files are
ZIPped. Next friday (or weekend) I'll produce my weekly stats using
2010.PRN and 2011,PRN (both are CSV, not Windows' definition of a PRN
file). When I'm done, I'll "PKZIP 2011.ZIP (roughly 500 KB) 2011.PRN
(roughly 2 MB, about 175 stocks and indexes) and move it to the
(maintained) archive.

Besides that, quotes are stored with a dot because programming
languages can process that immediately wiythout converting a "," to a
"." first. Any conversion is related to output only.

And I already wrote a one-off app to publish 2011's silly stats,
covering a few countries and stock exchanges. That would be required
if there was a shared data format... :-)
Post by Will Honea
the last time I looked I think I remember SQLLITE being
available for OS/2.
Just a few questions, I'm a Certified SQL nOOb: can field sizes be
adjusted (increased) dynamically, is there such a thing as
sizeof(COMPANIES.FULL_NAME), and can I store numbers with as many
digits as I like? Examples: 5.452, 5.45, 5.00. I'm aware this looks
like Rexx' format rules (SAY 5*2.50=12.50, not 12.5).

And a remark (minor detail): SQLLite, a tool, itself shouldn't become
a standard, because other user may use its latest version which may
not be available for everybody. I don't assume there's any need for
the latest version, but you'ld want to avoid a situation that
"Excel"/SQLLite replaces the word "spreadsheet"/underlying shared
data format.
Post by Will Honea
I would disregard the user's preference for a storage app
Right, mine's the first one I ever used. I.e. randomly selected, used
because some bloke offered free downloadable data in that readable
format. I just verified it was an existing format.
Post by Will Honea
concentrate on a simple definition of a good app agnostic
description
Post by Will Honea
of the major blocks (tables or spread sheet layout) just so the end
user
Post by Will Honea
knew the order and grouping of columns for each entity in the
database
Post by Will Honea
plus the necessary relationships between them and let it go.
That's also what I was thinking about. As long as an addition is
reasonable, it can be added. There's no need for a Perfect Final
Definition nor Object Oriented Dreams, including a complete set of all
nternational equivalents of strange local laws.

If I did mention user's preferences, think about settings to be
applied to the data. If there's a merger between A and B, I prefer to
delete all historical data instead of pretending A+B=AB. I don't want
rounded data, I prefer verified data, what's my "the" index, and so
on.
Post by Will Honea
Publish a detailed schema for what you use and let it go at that.
Again what I was thinking about, because thats's what a developer
would need.
Post by Will Honea
I have access to some NYSE feeds and they boil down to the
same final data.
No, they don't. Not counting rounding errors introduces by ICT. And:
YMMV, because there's more than one NYSE stock exchange. The French
stock exchange in Paris, for example.

Other errors, albeit they may fix it before you've seen it:

Point your browser to ...

http://www.euronext.com/landing/indexMarket-18812-EN.html?en

... and enter the ISIN-code GG00B1GHHH78 at the top of the screen
(Enter Symbol/Name/ISIN). What's the NYSE Euronext symbol of those
Volta Finance "shares"?

They (may) change historical data according to the "ratio method",
which is bogus (personal preference: bad luck, delete all historical
apples/peers-data).

ISIN-code NL0000009827 (DSM) random period: 8th of december upto and
including the 28th of december (14 trading days). Their current
historical end-of-day data:

Date opening High Low closing Volume
12/12/11 35.495 35.625 34.78 34.865 592,382.00
13/12/11 34.895 35.505 34.64 34.885 751,788.00
16/12/11 34.34 34.55 33.895 33.895 973,398.00
19/12/11 33.56 34.655 33.545 33.75 773,637.00

My daily verified data (only days with a different volume are
included):

DSM,D,20111212,35.495,35.625,34.78,34.865,602382
DSM,D,20111213,34.895,35.505,34.64,34.885,752888
DSM,D,20111216,34.34,34.55,33.895,33.895,1042398
DSM,D,20111219,33.56,34.655,33.545,33.75,787569

Let's verify the 19th of december, now having to use the Official
Price List. Aim the browser at:

http://www.nyse.com/opc

Or (PDF file of the 19th of december):

http://www.euronext.com/fic/000/068/476/684764.pdf

Page 7, column Centrale Markt: 773637. So the current historical
end-of-day data is right, but it used to be wrong. My 787569 was the
same original volume, representing verified (!) historical end-of-day
data. It was verified using yet another NYSE data source. Apparently
they "fixed" the data (column Meldingen, i.e. reported block trades),
at least once. In order to get this right, I'll have to store all of
my data, download it again later, and read newspapers in order to
verify it for the second or third time. Apparently in the end the
"newspaper" (PDF file) gets it right, but nobody knows when this is
sure. Seems like I need a PDF data extractor... :-/

Of course Yahoo! is wrong too (DSM.AS), due to rounding. They tell us
it's 773600 instead of 773637. BTW, using the PDF as a (third and
more!) data source is a problem because it also contains errors or
differences. BTW/2, AFAIK there's a law stating that a stock exchange
is allowed to publish wrong, verifyable data. So the NYSE is breaking
the law, IMhO.

Wtf cares? Well, you'll flame me if I report a so-called "verified"
volume of 787569 and today's version of the Official Price List
mentions a different value. How to make a data-verifying expert look
like an idiot, in one easy NYSE-step. Not to mentoin people using data
to support their decisions.
Post by Will Honea
The reply-to email in my headers is good - we might want to move
to email to continue the discussion.
I, ...

REXXTRY SAY X2C(Reverse('C6E6E256E6F6D296E6570413D6'))

... quit here, mentioning my data format:

Sorted (ticker, date) end-of-day stock quote data:

ticker (same as exchange), date (YYYYMMDD), open, high, low, close,
colume (index? 0!)

Data of listed companies and indexes:

ticker (same as above), full name for output files (length:20),
ISIN-code, profit J/N (Y/N, or logical: STOCK.my_EPS>=0?), group

group definitions:

I for indexes (J for the next stock exchange)
A for main index components (B for the next stock exchange)
M for mid cap index components (N for the next stock exchange)
S for small cap index components (T for the next stock exchange)
X for official OTC BB'ish crap (negative shareholders' assets)
Other: none ("", NULL)

In apps, possibly: directories (one per exchange), symbols to
include/skip, name of the main index per stock exchange in order to
know the (number of) trading days, names of indexes.

That's all, not counting apps which display a real-time stock exchange
index (ugly mode) or use specific market rules. The possible
improvements are clear, for one because there's hardly a different
point of view.

Even more on-topic: I don't think Text2PDF/2-apps are capable of
"converting" the PDF files, but I hope I'm wrong. Another solution is
to download data more often, and use the lowest volume. Not perfect,
of course. As a matter of fact, I was slightly surprised by the wrong
DSM volume. I'm 100% sure I verified it, if there was an error on the
19th of december I've fixed it, and I've never missed a day of
downloading verification data w.r.t. this stock exchange. NYSE's data
ain't perfect. Nor is Yahoo!'s, mainly because they use rounding.


--
A.D. Fundum
2011-12-29 23:53:27 UTC
Permalink
Post by A.D. Fundum
In apps, possibly: directories (one per exchange), symbols
to include/skip
In certain conditions, flags will do (A). Example: stock listed, but
no trades/quote yet. I use it to hide warnings (company listed, but no
quotes found)

Oops, and "date listed since" (B). Actually the first day of current
data. If your company is listed since today, I'm not missing
yesterday's quote.

A translated example:

if ((!strcmp(tickers[i],"BOEI\0"))&&(index[j]<20110830))
exists[i].ticker[j]=TRUE;

First there was no quote for those Boeing shares(situation A), for a
few years. Suddenly there was a transaction (or I deleted old data) on
the 30th of august. Before that day (situation B), data isn't assumed
to be missing (if exists[i].ticker[j]==FALSE then
display(fatalerror:missingdata).

So may data now looks like (first two BOEI-lines, please note this is
not the NY NYSE's volume):

BOEI,D,20110830,58.50,58.50,58.50,58.50,50
BOEI,D,20110831,58.50,58.50,58.50,58.50,0
(and so on, it's my ffirst recorded trade so far)

Before 20110830, I disabled the warning that I had no BOEI data (A).
After the first trade, I disable the warning that e.g. there's no
recod for BOEI for earlier dates (B).

BTW, this is a stock listed in USD (not checked, but it's obvious). I
use currencies as-is (full name including the currency: "Boeing $"),
so I didn't forget to mention that type of data. Perhaps I should use
it, but I don't.

BTW/2, you may want to distinguish between an index and a reinvestment
index. The use of the latter isn't common, but it's a type of index.
It's not included in my data, albeit I used it for one-offs.


--
Will Honea
2011-12-30 06:55:25 UTC
Permalink
Post by A.D. Fundum
I guess I was thinking about possible international issues and DBCS
company names. We may assume that 20 characters is more than enough to
hold "Bangkok Intl. Hotel", but I'm afraid you'll need more than 10
bytes to store "Krung Thep Mahanakhon Amon Rattanakosin Mahinthara
Ayuthaya Mahadilok Phop Noppharat Ratchathani Burirom Udomratchaniwet
Mahasathan Amon Piman Awatan Sathit Sakkathattiya Witsanukam Prasit
Intl. Hotel".
Most recent databases include the VARCHAR data type - variable length
strings which use basically 1 extra byte which stores either the lenght of
the string or a string termination character. Fixed length strings are
errors waiting to happen - or entry points for buffer-overflow security
exploits ;-)
Post by A.D. Fundum
Just a few questions, I'm a Certified SQL nOOb: can field sizes be
adjusted (increased) dynamically, is there such a thing as
sizeof(COMPANIES.FULL_NAME), and can I store numbers with as many
digits as I like? Examples: 5.452, 5.45, 5.00. I'm aware this looks
like Rexx' format rules (SAY 5*2.50=12.50, not 12.5).
Again, the commonly available data formats allow for storage in about any
spec you want. Some even include both storage and math using BCD (binary
Coded Decimal) which is invaluable for avoiding roundoff and truncation
error accumulation
Post by A.D. Fundum
And a remark (minor detail): SQLLite, a tool, itself shouldn't become
a standard, because other user may use its latest version which may
not be available for everybody. I don't assume there's any need for
the latest version, but you'ld want to avoid a situation that
"Excel"/SQLLite replaces the word "spreadsheet"/underlying shared
data format.
I've been in this business since the days of COBOL and Hollerith punch cards
so I rarely allow the i/o, storage mechanisms, and operating environments to
define the functional code. More times than I care to recall, I've written
or worked on applications that have survived language, hardware and
operating environments through multiple generations of each. Sometimes,
your Grandpa does have something to teach you ;-)

If you can modularize your functional code so that moving to a new
environment - or adding a new exchange/monetary system - reduces to changing
only the fragments which store and retreive the data then simply supply it
in the proper sequence to call and receive results you will have a much more
robust application even in the trivial cases.

The last job I held before retiring was for a multinational telecom company.
The group I led did customer billing, service scheduling, and inventory
control software and reporting world wide so your concerns are pretty
common. The key is identifying the essence of the data then abstracting the
actual processing such that it is isolated from the specific units of the
data. Then, regardless of the raw format, the process is to abstract the
values, process it, then map the result to the desired format (language,
units, etc.).
--
Will Honea
A.D. Fundum
2012-01-04 00:35:38 UTC
Permalink
Post by Will Honea
Most recent databases include the VARCHAR data type - variable
length strings which use basically 1 extra byte which stores either
the lenght of the string or a string termination character.
Sounds like a Rexx variable. Allright, so a need for a longer text
field doesn't imply yet another version of the data standard. The
preferred maximum length for different situations is an user's
setting, and apps can require certain length.

FTR: the name of investment funds can be quite long. Name of the
issuer, name of a region, name of an industry, other data ("Fantasy
Bank North-American High Yield Bonds Fund VI"). On the other hand the
size of the screen matters if you want to see my HTML table with 2
columns of data.
Post by Will Honea
Again, the commonly available data formats allow for storage in
about any spec you want.
Would you use a fixed number of digits after the decimal point (or no
digits and e.g. an integer recorded quote in 1:100,000 of a USD)? If
so, how many digits are in use, as far as you know? My worst case is
3, but I'm pretty sure some OTC BB stocks use far more than 3 digits.
Is there know limit? Or should we use integer and a number to divide
the integer (100->1.00). I'm asking this, because I'm not sure if
calculations or data conversions come with a huge speed penalty.
Post by Will Honea
I've been in this business since the days of COBOL and Hollerith punch cards
Hm, the oldest thing I've seen in the industry, related to computers,
was the production of "the" ticker tape in the early 90s.
Post by Will Honea
If you can modularize your functional code so that moving to a
new environment - or adding a new exchange/monetary system
- reduces to changing only the fragments which store and retreive
the data then simply supply it in the proper sequence to call and
receive results you will have a much more robust application even
in the trivial cases.
Yes, and it's important to be able to test e.g. a theory by using
unknown foreign data. Obtaining data is a problem, I don't know where
I can download data of the stock exchange of Milan, Italy. Let's
assume Yahoo! can provide that data. If so, my next problem is that my
customized apps contain hard-coded errors. Just changing a #define
INDEX "DJIA\0" isn't enough. Other indexes may be used in the same
file, it may assume 3 comparable types of indexes, and so on. My apps
are "modules", which can be used in a stand-alone environment. I'ld
expect using new data (I'm not using yet) is trivial too.
Post by Will Honea
The key is identifying the essence of the data then abstracting
the actual processing such that it is isolated from the specific
units of the data. Then, regardless of the raw format, the process
is to abstract the values, process it, then map the result to the
desired format (language, units, etc.).
Shall I scan my apps and data in order to construct an initial
proposal, including any problem I encounter? For example, I won't
include the Yahoo! symbol. Once that proposal is "approved", for
example after adding the Yahoo! symbol to a virtual database table,
perhaps you can translate that to both an actual SQLLite'ish database
and SQL data to obtain the data my apps need. You don't have to write
my apps, I mean a sort of How Do I or Programming Guide/Reference
samples. It doesn't have to be perfect, it's not about SQL.. Because
my knowledge of SQL is very limited. Once that's done, I (and you)
should be able to create my standard database and to rewrite my apps
to use the new data source.

FTR: let's assume I don't include Yahoo! symbols, because I don't use
those (yet). Any request to add those is fine with me, and I may add a
few quality requirements. For me it's a possible backup, but only the
daily data. Historical prices are rounded, daily (end-of-day) data
isn't. So my next proposal may include the Yahoo! symbols, and an
extra layer. Please note it's not my goal to call any shot, but we've
got to start somewhere. I'll try to ignore other issues, like sharing
data. I'm not sure if I want to provide data (data, corporate actions,
and so on) to people not contributing anything usefull or not sharing
any meaningfull app. I'm willing to include those issues, so it's
possible to exchange maintenance data, but it's not my priority.

FTR/2: a few of my areas of expertise are economic research, the stock
market in general, and computer science. All at a down-to-earth level,
I always try to use free data and resources instead of "cool" data,
and I like to avoid calculators. In my own language I should be able
to explain you why this ...

http://www.investopedia.com/terms/z/zero-sumgame.asp

... makes no sense at all. But if you expect a speech about the
changing role of the theta of a stock option, I'll try to ignore the
invitation. I also do care about data quality, as long as it's
possible to get in right in a reasonable way.

One final question, besides the one if I should endevour a first step
(let's continue that by email, if there's no general issue or remark
to be made in publiv): are there any license issues? Can a company,
researcher or an university use such a data model, or does a SQLLite/2
license claim too much (made up, like "you are not allowed to export
data for commercial use")? It's just a tool, it shouldn't stop any
underlying developments.


--
Will Honea
2012-01-04 19:51:17 UTC
Permalink
Post by A.D. Fundum
Would you use a fixed number of digits after the decimal point (or no
digits and e.g. an integer recorded quote in 1:100,000 of a USD)? If
so, how many digits are in use, as far as you know? My worst case is
3, but I'm pretty sure some OTC BB stocks use far more than 3 digits.
Is there know limit? Or should we use integer and a number to divide
the integer (100->1.00). I'm asking this, because I'm not sure if
calculations or data conversions come with a huge speed penalty.
You are wrapping yourself up in a conundrum that has persisted since I first
became involved with computers in the 1950's. There is no such thing as a
perfectly precise number system. Every known scheme is subject to two
errors: roundoff and truncation. The first is actually a manifestation of
the second in that it is a method of attempting to handle the latter and is
simply due to the fact that irrational numbers will exist so long as
partitioning of numbers exists - otherwise known as "division" or
"factoring". Many trees have been scarificed to efforts to quantify and
minimize the effects of these errors but so long as quantities (like 1/3 =
0.3333333... ad infinitum ) exist Phd candidates will continue to look for
an answer.

The current (pragmatic) solution is to use double precision floating point
as the storage container and for all numerical operations. Only the final,
end result is formatted for display purposes. Given the miniscule cost of
storage and the efficiency of computing elements, the speed cost is trivial
at best. So long as input data is assumed to be absolute and all computation
is done in the the units of the original data, error is reduced to the a
mechanical/hardware factor. This means that, for computational purposes,
the data is considered to be unit-less or abstract and only the presentation
of the end result is subjected to conversion error and cost.
Post by A.D. Fundum
Post by Will Honea
I've been in this business since the days of COBOL and Hollerith punch cards
Hm, the oldest thing I've seen in the industry, related to computers,
was the production of "the" ticker tape in the early 90s.
AH! Still wet behind the ears are you? I was contemplating retirement by
then ;-)
Post by A.D. Fundum
FTR: let's assume I don't include Yahoo! symbols, because I don't use
those (yet). Any request to add those is fine with me, and I may add a
few quality requirements. For me it's a possible backup, but only the
daily data. Historical prices are rounded, daily (end-of-day) data
isn't. So my next proposal may include the Yahoo! symbols, and an
extra layer. Please note it's not my goal to call any shot, but we've
got to start somewhere. I'll try to ignore other issues, like sharing
data. I'm not sure if I want to provide data (data, corporate actions,
and so on) to people not contributing anything usefull or not sharing
any meaningfull app. I'm willing to include those issues, so it's
possible to exchange maintenance data, but it's not my priority.
FTR/2: a few of my areas of expertise are economic research, the stock
market in general, and computer science. All at a down-to-earth level,
I always try to use free data and resources instead of "cool" data,
and I like to avoid calculators. In my own language I should be able
to explain you why this ...
http://www.investopedia.com/terms/z/zero-sumgame.asp
... makes no sense at all. But if you expect a speech about the
changing role of the theta of a stock option, I'll try to ignore the
invitation. I also do care about data quality, as long as it's
possible to get in right in a reasonable way.
One final question, besides the one if I should endevour a first step
(let's continue that by email, if there's no general issue or remark
to be made in publiv): are there any license issues? Can a company,
researcher or an university use such a data model, or does a SQLLite/2
license claim too much (made up, like "you are not allowed to export
data for commercial use")? It's just a tool, it shouldn't stop any
underlying developments.
You must have slept thru a couple of classes ;-) I can see that you have
missed a few points on design of Rational Data (RDMS). If the source of a
particular datum is significant, then that datum contains contains the
source. It is just one more qualifier like date, price, currency, etc.
which describes the point. Each element of the data record must contain,
either explicitly or by reference, all of the qualifiers significant to it's
interpretation.

A lot can be done to optimize processing and storage by the way you choose
to represent the particular pieces. Where the data are intended for machine
processing and contain repetitive information - say, company name - you may
want to store a small reference index into a separate table containing the
expanded information. OTOH, if you will be using the a spread sheet to view
the information, you will need a fairly complicated set of spread sheets to
translate those references into something human readable.

As for license issues on the database, there are none related to the data
itself. There are re-use and redistribution restrictions imposed by the
license under which you obtained the data but these are fairly obvious. The
price of IBM common shares is public but you can't get it from the NYSE feed
and redistribute it to all your clients without having a hoarde of lawyers
on your doorstep. You just have to be careful about redistributing data you
gather from each source. Check the TOS for each source - you may have to
point to the source and let each user download some of them for themselves.
Others are subscription services and impose a time delay and a few are
"proprietary" - it all depends on the source.

Even with something like DB2 or Oracle, there are no restrictions I am aware
of that prevent you from distributing a schema you develop. Nor is there
any restriction on the actual data itself as exported from those databases.
That's your product.
--
Will Honea
A.D. Fundum
2012-02-01 04:31:56 UTC
Permalink
Allright, so a need for a longer text field doesn't imply yet
another version of the data standard.
As already mentioned in the first line of this thread I've perhaps
started too late: :-)
"I'm quite busy at the moment"
I'm out of (this) business for an unknown period of time, because of a
"changed" (and broken, and ...) NYSE website. So I'm trying to rewrite
my software, if that's possible at all. Changed URLs, changed data,
changed file formats, the use of Flash instead of HTML, mixed use of
languages, broken links. And so on.


--
A.D. Fundum
2012-04-20 15:40:51 UTC
Permalink
Post by A.D. Fundum
Allright, so a need for a longer text field doesn't imply
yet another version of the data standard.
I'm out of (this) business for an unknown period of time, because
of a "changed" (and broken, and ...) NYSE website. Changed
URLs, changed data, changed file formats, the use of Flash
instead of HTML, mixed use of languages, broken links.
And so on.
Final update: I'm out of this business, because of the websites, If
their so-called "redesign" already is a well-known First Class
Disaster, the NYSE's current "We've temporarily restored [the website]
while we are improving performances on portfolio services" is
redefining First Class Distasters. Introduce the new one, restore the
former one because it's too redesigned, and the Next Redesigned
Website can be expected at any random moment. I'll give a +1 for the
restore, but basicly that just illustrates how "redesigned" it was.

Too much changes affecting daily data gathering (after a status quo
for several years), so introducing some standard is about as difficult
as creating WPS objects to achieve the same goals with a.o. using
drap&drop... :-/


--
Will Honea
2012-04-20 21:11:07 UTC
Permalink
Post by A.D. Fundum
Final update: I'm out of this business, because of the websites, If
their so-called "redesign" already is a well-known First Class
Disaster, the NYSE's current "We've temporarily restored [the website]
while we are improving performances on portfolio services" is
redefining First Class Distasters. Introduce the new one, restore the
former one because it's too redesigned, and the Next Redesigned
Website can be expected at any random moment. I'll give a +1 for the
restore, but basicly that just illustrates how "redesigned" it was.
Too much changes affecting daily data gathering (after a status quo
for several years), so introducing some standard is about as difficult
as creating WPS objects to achieve the same goals with a.o. using
drap&drop... :-/
Sorry to see you leaving. I've been AWOL for quite a while - spending too
darned much time with the doctors - but I'm slowly getting back to doing
something interesting beyond just staying alive.

I hope you aren't just shutting down the code. Simply putting it up on
dropbox or maybe Sourceforge "as is" would serve to get someone interested
in keeping the functionality alive. Heck, I've got plenty of space on my
own site to tuck away the archive if you want. Pull anything that you think
might be someone else's proprietary info, slap the gnu open source license
on the rest and see what happens.
--
Will Honea
Rich Walsh
2012-04-21 02:16:32 UTC
Permalink
slap the gnu open source license on the rest and see what happens.
Far better: put it under the Mozilla Tri-License (MPL/LGPL/GPL) so the
open-source code remains open-source while allowing it to be combined
with code that has different licensing. IMHO, the GPL stifles reuse.
--
== == almost usable email address: Rich AT E-vertise DOT Com == ==
Will Honea
2012-04-21 19:34:45 UTC
Permalink
Post by Rich Walsh
slap the gnu open source license on the rest and see what happens.
Far better: put it under the Mozilla Tri-License (MPL/LGPL/GPL) so the
open-source code remains open-source while allowing it to be combined
with code that has different licensing. IMHO, the GPL stifles reuse.
Probably the best license choice, Rick. tain't exactly rocket
science but what's there represents a lot of effort - even if the quote
sources are play games/inept. I know I find such gems to be invaluable.
--
Will Honea
A.D. Fundum
2012-04-22 13:39:31 UTC
Permalink
tain't exactly rocket science but what's there represents a lot of
effort - even if the quote sources are play games/inept. I know I
find such gems to be invaluable.
Right. My toy PCX2PRN (version not yet supporting colors already
released via Hobbes, but not in their money-section) is an example. If
was written to answer somebody's real question in some newsgroup, like
"I downloaded a wood futures graph, but where can I find the data they
used?!". Of course most of the effort is related to a lack of
programming skills.

I never considered using an Open Source concept nor a foreign
third-party license. It's a typical simple app indeed, some remake
isn't that hard, there's hardly anything to (c), some of the code may
be (avaidable) OS/2-specific, people wanting the data are looking for
free data anyway, and I'm not using GCC. One of the reasons for
releasing is was the fact that such an app is hard to find, if it
exists at all, and that the UI was good enough. IIRC I never released
the original Rexx-version, just the compiled ones.

Anyway, it apllies to the criteria. A toy, lots of effort, and the
guesstimated bogus it produces ought to be rejected by the Quality
Control department.


--
A.D. Fundum
2012-04-22 12:31:01 UTC
Permalink
Post by Will Honea
Post by A.D. Fundum
the NYSE's current "We've temporarily restored [the website]
while we are improving performances on portfolio services"
FWIW: I'm not blaming the NYSE for using stupid ICT methods, but [the
website] is NYSE's and I'm using it to indicate that I'm not relying
on data of the Tazkaristahian Stock Exchange, because of an invalid
airplane ticket located in Nigeria.
Post by Will Honea
I hope you aren't just shutting down the code.
No, but I'll probably cry (and be busy) for months as soon as I notice
the promised "improved performance".

I'm not a programmer as such, nor a programmer looking for work, but
my apps can be divided in a few categories. This a.o. assumes having
access to all required data, and it has to be based on stats or quite
interesting facts. If your friend can mention a website with easily
downloadable data of Jupiter's weather systems, I don't mind writing a
quick&dirty (portable Rexx) app which confirms the correlation between
the theoretical rain drop size and the main French exchange index.
That's sort of demand driven. Despite its informal "works for
me"-status, if it's interesting chances are I improve the app by using
e.g. HTML output and/or rewrite it using C to improve the processing
speed and the system's stability. It may be specific to a certain
country with specific rules.

An example of the above is an app, which is an "auction hunter". No
demand as such, but there were people caring about serious smallest
caps. The app shows all so-called market orders in bonds or stocks
using a so-called call auction trading system. So we're looking for a
sale without buyers, which fits a vaule investor's strategy. Albeit
the realized discount itself often attracts buyers the next day. An
example of a "new" app could be an app which shows if you should use a
MA-199 instead of an usual MA-200.

My own apps, with HTML-output, tend to be aimed at providing a quick
view at a(ny) market. Assuming having access to German data, I should
be able to write an educated article about any German stock and I
should be able to select stocks to (not) buy. Those apps are often
based on statistics, e.g. last month's top-100 gainers. The apps can
be demand driven too. I don't look at stock quote graphs at all, but
one of my apps shows small stock quote graphs of all stocks because
somebody once asked for it. An example of a "new" app could be a list
of stocks where the MA-40 line crossed the MA-200 line, so the
computer instead of you compiles a short-list (i.e. a quick view at
any market, if one would use an investing strategy based on moving
averages).

Let's summarize the above, and claim that the software in question in
based on statements (Jupiter's weather systems predict our future far
better than a gorilla, or "I always buy stocks when the DJIA lost over
10% in 4 trading days"). Or it can be based on statistics (volatility,
today's gainers, the RSI, and so on).

This also explains why I do care about data quality. I'm using it to
make decisions, and I like to get verifyable results right. Of course
I also have a few generic apps, stock exchange-specific downloaders
and matching verifying apps.

Basicly all of the above is based on Rexx apps, at least when it came
to prototyping, and anybody clould write those apps. The underlying
question is probably more (c)-worthy, like asking if using a MA-201 is
better than using a MA-200. If it takes too much time, for example, or
it isn't appealing to me, I'll probably ignore those questions.

I guess a final category of apps, at least for now, are apps which are
based on my "Jupiter weather systems"-based research. The output may
be at a HTML (simple Netscape!) quality level, but it'll be useless to
anybody else. Too unclear, too complicated, too easy, looking like
"complete loose stool water, arse-gravy of the worst kind" (next
week's rather silly prediction, the HTML- table shows it :-)):
^ASCX.AS outperforms the ^AEX, the ^AMX canl also outperform the ^AEX
), ... Then it's better to operate some website displaying the
information than to share the underlying app. If I would share such
apps, chances are having OS/2 is an requirement, for one because I
don't want users complaining that they cannot import the data in Excel
retry 2012.

So apps I'll write are or a random collection of apps ever written and
worth not deleting immediately, or apps are inspired by demand.
There's no English investing-related newsgroup or forum I'm reading,
but so lang as the NYSE doesn't release yet another "redesign", I'm
always willing to look at interesting, appealing requests for
quick&dirty apps. In general there's a lack of demand, the best and
most obvious example still is a lacking (other value- or)
index-displaying eCenter widget. The CPU load and history of the CPU
load seems to be more important... ;-)

There's hardly a legal issue, at least with my apps. It's likely that
you are legally allowed to upload (but not to download) a (c) movie,
while I'm legally allowed to download (but not to upload) a (c) movie.
Most apps are quick&dirty works-for-me-prototypes or poor man's
solutions, e.g. an app SAY'ing the free realest-time stock exchange
index every X seconds during trading hours.

Already having published some apps, I know problems are 'em being
quick&dirty. If you aren't looking for a specific app, a Rexx "script"
is too complicated if you don't already have Rexx (and wget, and ...).
Ignoring the bugs and looks, the second international problem is
having access to the data. Ask a question about the main German stock
exchange, and I don't know its rules nor where German people typically
obtain their (free: requirement) data. Some Newspaper's Website, the
Boerse itself, ...? The third issue is the data format, but typically
the author of an app can enforce some text file format.

Next, and most likely to happen, is outdated data or outdated
software. It happens very, very often that "something" changes. Like
NYSE websites, or a symbol of the main index. My apps written in C use
a #define to define the main index (which defines a trading day, and
an allowed volume of 0 if high<>low), but essentially that means an
user has to have a C compiler to change such a value. At least if you
want to use my apps as-is. A proper app could show a lists of indices,
but of course that adds data that can be outdated soon, and I don't
really know the symbol(s) nor name(s) of the Hungarian index.

The auction hunter is an example of an outdated app. It (still) should
work fine, but ... the ISIN-codes are hard-coded to keep the app
itself simple (i.e. it doesn't update the data dynamically, instead it
assumes the list is right)
Post by Will Honea
Pull anything that you think might be someone else's proprietary
info
I already did that. Some apps aren't good enough to release, like the
poor man's realest-time stock exchange index monitor, which is hardly
more than:

...
PARSE VAR htmlfile . 'LAST_PRICE">' index '<' .
IF DataType(Left(index,1))='NUM' THEN DO
SAY Date() Time() Strip(index)
...

To be worth releasing, it should at least look like an eCenter-widget
I'm advocating myself. ;-)

An app to download stock quotes is not fully releasable for generic
use in specific countries, and its data model is outdated quite soon.
It'll work, but only for a limited period of time. About half of the
apps producing statistics only have a use in certains situation. For
example, I developed a simple index to show the liquidity of listed
stocks, and a simple app to display an intraday volatility. Yet
another tool to spot dump&dump-stocks, but that's just one use.
Post by Will Honea
see what happens.
Assuming the NYSE didn't rerelease redesigned websites last year, I'm
nearly almost willing to look at answering questions, i.e. demand,
which require some quick&dirty Rexx app to provide an underlying
answer or to check or proof an interesting statement. But users may
experience the problems this thread was about, I'll be using my data
and my data format. "I think it works for me", that's about it.

Besides that, an eCenter-widget perhaps shouldn't load a website every
10 seconds. It's not unlikely that the functionality will disappear
because of the system load it causes, we may look like hackers, and we
don't look at the advertisements.


--
Will Honea
2012-04-22 18:41:29 UTC
Permalink
Post by A.D. Fundum
If your friend can mention a website with easily
downloadable data of Jupiter's weather systems, I don't mind writing a
quick&dirty (portable Rexx) app which confirms the correlation between
the theoretical rain drop size and the main French exchange index.
Browse here:

http://moneytide.com/hans/index.asp

That will give you a clue to what he's doing. I've known Al since our USAFA
days - roomed with him at pilot training, flew a bunch of missions with him
in Vietnam, attend grad school with him (EE), then taught with him back at
USAFA. He's up in Denver so we keep in touch. Sounds like the two of you
might hit it off ;-)

From what you describe here and from earlier messages it sounds like you
have what amounts to a library of useful functions. In that sense, it
really doesn't matter really what language you use nor is the code quality
significant if the source code is available. Should a few active users form
they will eventually settle on some language (or languages) informally
anyway. This may also provide you with some useful feedback/help from those
interested in a particular locality saving you a bunch of research.

The biggest reason to attach a license - be it GPL, Mozilla, or some home
brew - is to prevent someone else from grabbing your work then claiming
copyright on it so that you and everyone else is locked out of it. The
world is full of folks who seem to have nothing better to do than screw up
things for the rest of us ;-(;
--
Will Honea
A.D. Fundum
2012-04-23 14:10:05 UTC
Permalink
Post by Will Honea
http://moneytide.com/hans/index.asp
And I wasn't trying to be facetious. Sort of returning the favour, by
sorting a chaotic system. If weather is a chaotic system, mine's
rougly based on binary logic. If the maximum temperature in Austin,
Texas, is 120F today, it's most likely that we're talking about the
summer. If we combine a few of those logical bits, the market can be
divided in (undefined) "groups". In group one the season is summer,
it most likely it's raining and it's a cloudy day. Let's assume that's
true, at least on average.

When I sort all stocks accordingly, e.g. on the 13th of march 2009,
I'm assigning labels to the X-axis of a graph. AMG is #1, SNS Reaal is
#2, and so on. Available for a short while:

http://home.uni-one.nl/m1/ranking.html

The table near the end is an attempt to list the most surprising moves
in the list/market, as opposed to lists of gains and losses in %.

After 8 monts, I now randomly selected files ending with 13
(yyyymmdd), the Y-axis represents the ROI in %. The ROI is not just
the ROI of buying #1, but a virtual mix which mainly invested in #1, a
bit in #2, and less in #3. It are averages of averages, applied to
smoothen the graph:

Loading Image...

The left side of the graph, roughly 95 stocks, is divided in 4 groups.
You cannot see the boundaries of the groups (I could), but in general
it can be said that stocks on left side of the graph, inlcuding 3
"mountains", were the ones to buy. The left side of the graph cleary
outperformed the market (main index:+49.89%), and without applying any
other selection, with embedded known errors and accepting it having to
last 8 months, it almost always outperforms the index with the best
performance of +75.50% too). As a matter of fact, this is the best
short-listcreators I've ever seen, and its reproducable so-called
"hitratio" is, according to a scientist, "unlikely". Essentially I'm
creating (some) "order" in the "chaos".

So far this is, of course, on-topic, because ...
Post by Will Honea
The biggest reason to attach a license - be it GPL, Mozilla, or
some
Post by Will Honea
home brew - is to prevent someone else from grabbing your work
then claiming copyright on it so that you and everyone else is
locked
Post by Will Honea
out of it. The world is full of folks who seem to have nothing
better to
Post by Will Honea
do than screw up things for the rest of us ;-(;
... I don't have an intention to launch a website, but the involved
underlying OS/2 apps are protected because I won't release the source
code, I won't include the required book-sized manual, and - if needed
- there's old evidence that's it's my idea (this very message, for
example).

If I'm going to release this type of (c)-worthy apps, PCX2PRN is
another example (I wouldn't mind a better remake, the UI isn't
perfect), it'll be the EXE and maybe some sample files likes the ones
above. IRL I've distributed PCX2PRN's output instead of telling
somebody to install OS/2 and use it. This kind of apps has hardly any
use, but it may inspire another idiot and it's showing that we've got
more than just Yahoo!-downloaders.
Post by Will Honea
Sounds like the two of you might hit it off ;-)
I'm afraid it'ld be a chaotic meeting... :-P
Post by Will Honea
From what you describe here and from earlier messages it sounds
like you have what amounts to a library of useful functions.
Indeed, albeit I've deleted most (Rexx) apps because the underlying
question was rare, and/or because a rewrite is not that hard, and/or I
answered by providing the multiplatform Rexx source code. So the
number of apps isn't that impressive.

Releasing Rexx apps, as-is, is no problem. The main issue is that
they'll be unclear, e.g. the auctoin hunter doesn;'t output anything
if there's no market order at the moment. So if you'ld download and
try it, it may be a disappointing experience.

So there are (working examples, a few EXE-based apps, a few Rexx-apps
(not "compiled") and apps written in C. Typically C is used for
processing speed, Rexx is used to beautify the output.
Post by Will Honea
In that sense, it really doesn't matter really what language you
use nor is the code quality significant if the source code is
available.
Hmz. Earlier I could describe my data model (and possible add-ons),
but I already told that'll take too much time as soon as the NYSE
decides to rerelease their "redesigned" website. But if I won't do
that, I indeed could release all qualifying apps. As-is, i.e. with a
few works in a foreign language, no tiny nor book-sized manuals, no
comments, known errors, requiring WGET and/or GDIAGRAMM, and so on.

If I release an EXE without (c), I'll actually release the C source.
Why? Because you have to adjust it anyway. If I haven't #defined my
main index to be "AEX\0", the code has to be adjusted. The main index
determines what a trading day is, other indici are included in code
too (e.g. to allow a volume of 0 in case high<>low), and all apps
assume my data format.

BTW, this is an example. If you somehow don't understand the essence
of C source and I don't write a manual, it'll be as useful as the
so-called book The Da Dinvi Code, representing Stephen Fry's
arse-gravy of the worst kind:

http://home.uni-one.nl/m1/liqiuditeitsindices.html

Looking at the left table, e.g. Royal Dutch Shell at #9, it indicates
that buying or selling Shell shares ought to be easy. The index (I
made up) is well above about 40-50, and close to the maximum of 100.
The right table lists Shell at #86. The difference is that the
intraday ratio high:low doesn't often exceed 1.02:1.00. Even if you've
never seen this market before, it's very likely that Shell isn't a
trader's toy, nor is it likely that it's a "cheap" share, nor is it
likely that it's some stupid pump&dump-share. The opposite is also
true, e.g. Pharming. A small biotech company, but it seems many people
try to buy the share at $0.195 and try to sell it at $0.200. A larger
company, ING with its banking&insuarance (AMG's M is metal, not
motors), combines the best of both world. A lot of people are buying
and selling in the company (general liqiudity, expressed in the left
column), and almost every day the margin between highs and lows (right
column) is enough to facilitate e.g. daytraders. So without knowing a
market, you'ld be able to tell something about most listed, unknown
companies.

You'll need to understand the C source in order to understand and
aplly it, because (already assuming the availablity of the right data
in the right format) the output of the EXE is pretty useless:

20110421 20120420
...
INGA 99.61089325 91.85906982 49539670016.00
...
PHARM 91.43968964 87.17013550 122529136.00
...
PHIA 100.00000000 65.36965179 20264263680.00
...

Go figure.
Post by Will Honea
may also provide you with some useful feedback/help from those
interested in a particular locality saving you a bunch of research.
My question is: should I release it to Hobbes? "It": working example,
some EXEs, some C files with beautifying CMD files, some CMD files.

A disadvantage is that's not clear at all. It's not aimed at lazy
end-users at all. At least 2 of the Rexx apps may stop working as soon
as the NYSE Euronext website changes. Data files are outdated pretty
quickly. The poor man's realest-time index monitor sort of assumes GMT
+1, so in New York it'll often only work between 9:30 AM and 11:30 AM,
in e.g. France that's between 13:30 PM and 15:30 PM (shared opening
hours, and both between 9:30 AM and 5:30 PM, local time).

Another possibility is to post it in a newsgroup (c.o.o.p.m). The
number of apps is limited, and so is the number of lines. Line-length
is an issue, I won't attach binaries, but often the source has to be
adjusted anyway (e.g. adjusting or removing the names of indices).
This excludes the EXE, especially the chaotic one (PCX2PRN is already
released). I think I'm preferring this option, FWIW. There's no need
to email, somebody not caring has to skip a few messages (or can flame
me for the programming style with poor UIs), and it's archivable by
e.g. you (I'll exclude archiving by google).

Finally I could email it to anybody, but a prepared package will only
be available for a short while (due to not maintaining such a
distributable package). If dumping it in c.o.o,p.m is considered to be
slightly wrong, I'll prefer this. Apps that aren't really
distributable won't be distributed by email too.

So: C.o.o.p.m? If so, I'll start looking at compiling a package with
qualifying apps, and so on.

Finally/2: I already didn't mind possibly embedding some apps in a
"toolbox", IIRC discussed earlier. Assuming a (free) shared (free)
data model, and perhaps shared (free) data, adding (shared) apps to
use this data just makes sense. If somebody else already wrote a
Yahoo! quotes downloader, there's no need that everybody shall write
such a generic beast. So long as you have an OS/2 system, you'ld be
able to use all of my available apps as-is. If you don't have an OS/2
system, you'll be able to use/embed some of my apps/source/ideas/code
(Rexx apps, or my Rexx apps rewritten as a COBOL for DOS 3.0 app).


--
A.D. Fundum
2012-04-25 15:21:19 UTC
Permalink
Post by A.D. Fundum
Releasing Rexx apps, as-is, is no problem. The main issue is
that they'll be unclear, e.g. the auction hunter doesn't output
anything if there's no market order at the moment. So if you'ld
download and try it, it may be a disappointing experience.
The poor man's stock exchange index monitor. It's simple, it just
downloads a webpage about every 15 seconds. WGET is required. The
default index is Paris' CAC-40, and it'll work best between 8:56 and
17:40 CET (trading hours 9:00-17:35) and an overlapping local time. So
in New York it'll often work between 9:30AM and 11:30 AM, IIRC. It's
just a website parser, and the timestamp is the local time. Please
don't use it all the time, and please use a download fequency that
matches your free needs.

Of bloody course adding more indices (i.e.e free data sources with a
realest-time index value) is a generic improvement, just like
migrating it to an eCenter widget with graph and settings. Other
improvements are a trading day calender, logging, comparing it with
tick data (also available for free, but not available before the
night), detection of data errors, synchronizing the updates with the
real interval (if an index is calculated every 15 minutes, then
updating it about every 15 seconds is useless), synchronizing of the
local time with the asssumed stock index clock, online software
version check, and so on.

As you can see the interval is set at 14 seconds by default, but the
gap between 21 and 40 is 19 seconds with an ADSL2 connection:


25 Apr 2012 16:35:50 - 3.233,49
25 Apr 2012 16:36:06 - 3.235,14
25 Apr 2012 16:36:21 - 3.234,65
25 Apr 2012 16:36:40 - 3.234,65
25 Apr 2012 16:36:55 - 3.235,09


The code is not OS/2-specific, so one could always try to use it at
the office with some Rexx interpreter for e.g. Windows and any (GNU)
WGET for Windows. I've adjusted the length of the lines to avoid
wrapping, so copy&paste should work. The format of the index values is
European, but changing anything is quite easy. I'm not in the USA, so
no length disclaimers nor warnings, and the data is (or can be)
delayed. I also don't know if this is the fastest nor most reliable
source for free index data (for Belgium, France, Portugal and The
Netherlands).

BTW, didn' I mention the ^ASCX.AS would outperform the ^AEX? Let's
stop with that one. AEX -0.21%, ASCX -0.10%, at the moment. BTW/2,
I've decided to not publish the source of DAYS.EXE (@ Hobbes). I've
already deleted the source of this distributed quick&dirty version,
it's more versatile than my own, untranslated version and a remake is
very easy.


--

/* Monitor.CMD */

CALL RxFuncAdd 'SysCls','RexxUtil','SysCls'
CALL RxFuncAdd 'SysFileDelete','RexxUtil','SysFileDelete'
CALL RxFuncAdd 'SysSleep','RexxUtil','SysSleep'

SIGNAL ON Halt

/* AEX NL0000000107 */
/* BEL20 BE0389555039 */
/* CAC40 FR0003500008 */
/* PSI20 PTING0200002 */

isin='FR0003500008'

CALL SysCls
SAY
mep=''
IF isin=='NL0000000107' THEN mep=2
IF isin=='BE0389555039' THEN mep=3
IF isin=='FR0003500008' THEN mep=1
IF isin=='PTING0200002' THEN mep=5
IF mep='' THEN DO
SAY 'Unknown or invalid ISIN-code:' isin
SAY
EXIT
END

ua='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1;'
ua=ua||'en-US; rv:1.5b)"'
url='"http://www.euronext.com/trader/summarizedmarket/stocks'
url=url||'-2634-NL-'||isin||'.html?selectedMep='||mep||'"'

IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END
'@WGET -q -OINDEX' ua url
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
bestand=CharIn('INDEX',1,28672)
CALL CharOut 'INDEX'
CALL SysFileDelete 'INDEX'
PARSE VAR bestand . 'LAST_PRICE">' index '<' .
index=Strip(index)
IF DataType(Left(index,1))='NUM' THEN DO
SAY Date() Time() '-' Translate(index,'.,',',.')
END
END

IF Date('B')//7<5 THEN DO
DO WHILE Time('S')<63600
DO WHILE Time('S')>32160
CALL SysSleep 14
'@WGET -q -OINDEX' ua url
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
bestand=CharIn('INDEX',1,28672)
CALL CharOut 'INDEX'
CALL SysFileDelete 'INDEX'
PARSE VAR bestand . 'LAST_PRICE">' index '<' .
index=strip(index)
IF DataType(Left(index,1))='NUM' THEN DO
SAY Date() Time() '-' Translate(index,'.,',',.')
END
END
END
END
END

IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END

EXIT

Halt:
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END
EXIT
A.D. Fundum
2012-06-22 17:06:32 UTC
Permalink
Post by A.D. Fundum
The poor man's stock exchange index monitor.
Minor change, FWIW: add a SysSleep before the second last EXIT, so it
will alyways display at least one value in ache timezone, even if the
WPS is used to launch it:


---

CALL SysSleep 2 /* Add this line, the value 2 may be increased /*

EXIT

Halt:
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END
EXIT
A.D. Fundum
2012-08-09 18:42:12 UTC
Permalink
Post by A.D. Fundum
The poor man's stock exchange index monitor.
FTR: as such this app won't work anymore, due to a "redesigned
website" (i.e. a move to yet another, different, slower website:
nyse.com).


--
A.D. Fundum
2012-08-12 23:01:11 UTC
Permalink
Post by A.D. Fundum
FTR: as such this app won't work anymore, due to a "redesigned
website" (i.e. a move to yet another, different, slower website
Disclaimers: requires WGET's WGETSSL.EXE instead of WGET.EXE. And it's
a quick update, so I haven't even tried if the data used is undelayed,
i.e. as "real-time" as possible for free.

Again, adjust at least the (ISIN-code to select an index and adjust
the) values 32160 (8:56am) and 63600 (7:40pm) if your TZ doesn't match
the opening hours of the stock exchange. If you get it right, it's
output (with local date and time) could look like:

13 Aug 2012 08:59:38 - 3435.62
13 Aug 2012 08:59:58 - 3435.62
13 Aug 2012 09:00:21 - 3452.10
13 Aug 2012 09:00:40 - 3460.79
13 Aug 2012 09:01:01 - 3458.34
13 Aug 2012 09:01:19 - 3458.34
...
--
/* Poor Man's Stock Exchange Index Monitor */

CALL RxFuncAdd 'SysCls','RexxUtil','SysCls'
CALL RxFuncAdd 'SysFileDelete','RexxUtil','SysFileDelete'
CALL RxFuncAdd 'SysSleep','RexxUtil','SysSleep'

SIGNAL ON Halt

/* AEX NL0000000107 */
/* BEL 20 BE0389555039 */
/* CAC 40 FR0003500008 */
/* PSI 20 PTING0200002 */

isin='FR0003500008'

CALL SysCls
SAY
mep=''
IF isin=='NL0000000107' THEN mep='XAMS'
IF isin=='BE0389555039' THEN mep='XBRU'
IF isin=='FR0003500008' THEN mep='XPAR'
IF isin=='PTING0200002' THEN mep='XLIS'
IF mep='' THEN DO
SAY 'Invalid ISIN-code:' isin
SAY
CALL SysSleep 2
EXIT
END

ua='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-'
ua=ua||'US; rv:1.5b)"'
url='"https://europeanequities.nyx.com/nl/nyx_eu_listings/real'
url=url||'-time/'||'quote/'||isin||'-'||mep||'"'

IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END
'@WGETSSL.EXE -q --timeout=6 -OINDEX' ua url
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
file=CharIn('INDEX',1,8192)
CALL CharOut 'INDEX'
CALL SysFileDelete 'INDEX'
PARSE VAR file . 'id="lastPriceint">' prefix '<' file
PARSE VAR file . 'id="lastPricefract">' suffix '<' .
index=Strip(prefix)||Strip(suffix)
IF DataType(Left(index,1))='NUM' THEN DO
SAY Date() Time() '-' Translate(index,'.,',',.')
END
END

IF Date('B')//7<5 & Time('S')>32160 THEN DO
DO WHILE Time('S')<63600
CALL SysSleep 14
'@WGETSSL.EXE -q --timeout=6 -OINDEX' ua url
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
file=CharIn('INDEX',1,8192)
CALL CharOut 'INDEX'
CALL SysFileDelete 'INDEX'
PARSE VAR file . 'id="lastPriceint">' prefix '<' file
PARSE VAR file . 'id="lastPricefract">' suffix '<' .
index=Strip(prefix)||Strip(suffix)
IF DataType(Left(index,1))='NUM' THEN DO
SAY Date() Time() '-' Translate(index,'.,',',.')
END
END
END
END

IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END

CALL SysSleep 2

EXIT

Halt:
IF Stream('INDEX','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'INDEX'
END
EXIT
A.D. Fundum
2012-04-25 16:44:44 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
Releasing Rexx apps, as-is, is no problem. The main issue is
that they'll be unclear, e.g. the auctoin hunter doesn;'t output
anything if there's no market order at the moment. So if you'ld
download and try it, it may be a disappointing experience.
The auction hunter, which may stop working as we speak because it
seems the "redisgned" website used is about to be rererererereleased.
It should work with any (delayed) free order book data provider with
websites that can be parsed. Again, WGET is required. Liking and using
the data are different stories. I can think of a few improvements, it
ain't a cOOl one-button multimedia stock trading app. The sample
includes a few bonds, and it may run for a while due to the number of
websites to be downloaded.

An example of its output, if any:


Order book 27/47 of: RBS CAP FUND TRVII (RBST7, US74928P2074)
Last: 12,00
1 500 10,10 At Best 1 1
1 500 10,09 13,69 5 1
0 0 - 13,70 33 1

Order book 40/47 of: RO MMB 0%13 (XS0208678846)
Last: 127,40
0 0 - At Best 16000 1
0 0 - 127,40 20000 1


--

/* Auction.CMD */

CALL RxFuncAdd 'SysFileDelete','RexxUtil','SysFileDelete'

SIGNAL ON Halt

isin.0=47

isin.1='NL0010070777 ' /* 1NOMY */
isin.2='CWN001011006' /* AMIGO */
isin.3='DE0008600966' /* BACFP */
isin.4='US0737301038' /* BEAM */
isin.5='NL0000285278' /* BEVER */
isin.6='US0970231058' /* BOEI */
isin.7='US1667641005' /* CHTEX */
isin.8='LU0006013907' /* CIF */
isin.9='XS0138973010' /* DEPCF */
isin.10='NL0009733351' /* DICO */
isin.11='NL0000288017' /* EAT */
isin.12='US3448491049' /* FOOTL */
isin.13='NL0000440824' /* GROHA */
isin.14='NL0000358158' /* HITT */
isin.15='ANN4327C1303' /* HUNDP */
isin.16='NL0009391242' /* HYDRA */
isin.17='US4592001014' /* IBMA */
isin.18='US4595061015' /* IFF */
isin.19='US5658491064' /* MROIL */
isin.20='NL0009822014' /* NSE */
isin.21='US6745991058' /* OCPET */
isin.22='US7134481081' /* PEP */
isin.23='NL0000378669' /* PORF */
isin.24='NL0009875483' /* PREVA */
isin.25='US74928K2087' /* RBST5 */
isin.26='US74928M2044' /* RBST6 */
isin.27='US74928P2074' /* RBST7 */
isin.28='NL0000288736' /* ROLPR */
isin.29='NL0000292720' /* ROPEQ */
isin.30='NL0000388684' /* UNCC7 */
isin.31='NL0000388742' /* UNCP6 */
isin.32='NL0000388726' /* UNCP7 */
isin.33='US9078181081' /* UNPA */
isin.34='NL0009082486' /* VALUE */
isin.35='US92343V1044' /* VERIZ */
isin.36='NL0000394567' /* WEG */
isin.37='XS0204176332'
isin.38='XS0204176415'
isin.39='XS0167896660'
isin.40='XS0208678846'
isin.41='XS0192658549'
isin.42='XS0188829229'
isin.43='XS0192658382'
isin.44='XS0366178290'
isin.45='XS0239558819'
isin.46='XS0229443550'
isin.47='XS0303002389'

ua='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1;'
ua=ua||'en-US; rv:1.5b)"'
DO i=1 TO isin.0
url='"http://www.euronext.com/trader/advancedmarket/advanced'
url=url||'market-2594-NL-'||isin.i||'.html?selectedMep=2"'
'@WGET -q -OAUCTION.TMP' ua url
IF Stream('AUCTION.TMP','C','QUERY EXISTS')<>'' THEN DO
grootte=Stream('AUCTION.TMP','C','QUERY SIZE')
alles=CharIn('AUCTION.TMP',1,grootte)
CALL CharOut 'AUCTION.TMP'
CALL SysFileDelete 'AUCTION.TMP'
IF Pos('>At ',alles,1)>0 THEN DO
lagen=0
DO j=0 TO 4
IF Pos('bidParticipants'||j,alles,1)>0 THEN DO
lagen=Max(j+1,lagen)
END
IF Pos('askParticipants'||j,alles,1)>0 THEN DO
lagen=Max(j+1,lagen)
END
END j
IF lagen>0 THEN DO
PARSE VAR alles . 'width="70%">' naam '</td>' rest
PARSE VAR rest 'LAST_PRICE">' laatste '<' . 'Mnemo' rest2
PARSE VAR rest2 . '&nbsp;' symbol '<' rest
symbol=Strip(symbol)
symbool='('
IF symbol<>'' THEN symbool=symbool||symbol||', '
CALL CharOut '','Order book' Format(i,3)||'/'||isin.0
SAY ' of:' naam symbool||isin.i||')'
IF DataType(laatste)='NUM' THEN DO
SAY 'Last:' Translate(laatste,',','.')
END
IF DataType(laatste)<>'NUM' THEN DO
SAY 'Last: -'
END
DO j=0 TO lagen-1
split='bidParticipants'||j||'">'
PARSE VAR rest . (split) data1 '<' rest
split='bidVolume'||j||'">'
PARSE VAR rest . (split) data2 '<' rest
split='bidPrice'||j||'">'
PARSE VAR rest . (split) data3 '<' rest
split='askPrice'||j||'">'
PARSE VAR rest . (split) data4 '<' rest
split='askVolume'||j||'">'
PARSE VAR rest . (split) data5 '<' rest
split='askParticipants'||j||'">'
PARSE VAR rest . (split) data6 '<' rest
DO WHILE Pos(',',data2,1)>0
PARSE VAR data2 deel1 ',' deel2
data2=deel1||deel2
END
IF DataType(data3)='NUM' THEN DO
data3=Translate(data3,',','.')
END
IF DataType(data4)='NUM' THEN DO
data4=Translate(data4,',','.')
END
DO WHILE Pos(',',data5,1)>0
PARSE VAR data5 deel1 ',' deel2
data5=deel1||deel2
END
regel=Right(data1,3) Right(data2,6)||' '
regel=regel||Right(data3,7) Right(data4,9)||' '
regel=regel||Right(data5,6) Right(data6,3)
IF Strip(regel)<>'' THEN SAY regel
END j
SAY
END
END
END
END i

EXIT

Halt:

IF Stream('AUCTION.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'AUCTION.TMP'
END

EXIT
A.D. Fundum
2012-08-29 12:21:12 UTC
Permalink
Post by A.D. Fundum
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
Releasing Rexx apps, as-is, is no problem. The main
issue is that they'll be unclear, e.g. the auctoin hunter
doesn't output anything if there's no market order at
the moment. So if you'ld download and try it, it may
be a disappointing experience.
The auction hunter, which may stop working as we speak
because it seems the "redisgned" website used is about
to be rererererereleased.
I've attached the "rererereredesigned" app I've posted earlier. It
requires WGETSSL.EXE and assumes the use of
"europeanequities.nyx.com"-markets. Below is an example of its output.
If the value of Goodyear shares is considered to be about EUR 9.40,
then this could be an opportunity for a value investor (or a trader)
to buy 319 Goodyear shares at EUR 8.70 (or 320 at EUR 8,71). That's a
discount of about 7.45%, i.e a possibly attractive void, thanks to the
seller's market order.

Order book 30/71 of: GT (BE0004359916)
Last: 9.40 (27/08/2012)
1 130 MKT MKT 450 1
1 996 8.70 9.25 20 1
1 327 7.50 9.60 300 1
9.80 300 1
9.89 250 1
10.00 300 1
10.20 300 1
10.40 300 1
10.45 315 1
10.50 45 1

Order book 31/71 of: HITT (NL0000358158)
Last: 7.00 (29/08/2012)
1 98399 7.00 MKT 35 1
2 10200 6.60 7.00 3500 1
2 72427 6.56 7.15 5000 1
2 130000 6.55 7.25 2000 1
2 40672 6.54 7.26 7148 2
1 24278 6.45 7.40 210 1
1 189 6.36 7.44 255 1
1 1000 5.30 7.45 2000 1
7.50 100 1
7.99 1500 1

Order book 57/71 of: SER (BE0003677888)
Last: 12900.00 (29/08/2012)
1 2 12851.00 MKT 2 1
1 2 12850.50 12850.00 11 1
1 1 12850.01 13200.00 1 1
3 3 12850.00 13249.99 1 1
1 1 12803.00 13250.00 5 2
1 1 12801.00 13500.00 10 5
5 8 12800.00 13613.00 1 1
1 1 12755.00 13750.00 1 1
1 1 12600.00 13842.00 1 1
1 5 12502.00 14000.00 2 1

Order book 58/71 of: SOLV (BE0003545531)
Last: 99.00 (28/08/2012)
1 100 98.55 MKT 30 1
2 1250 98.50 99.98 100 1
1 300 98.25 101.85 100 1
1 500 98.00 102.00 265 2
1 50 97.50 104.88 75 1
2 250 97.00 104.90 50 1
2 192 95.00 104.95 697 1
1 100 94.00 105.00 675 3
1 76 93.40 105.50 500 1
2 210 93.00 106.00 100 1


--
/* Auction.CMD */

CALL RxFuncAdd 'SysFileDelete','RexxUtil','SysFileDelete'

SIGNAL ON Halt

isin.0=71

isin.1='NL0010070777 XAMS' /* 1NOMY */
isin.2='CWN001011006 XAMS' /* AMIGO */
isin.3='FR0004166155 XBRU' /* ART */
isin.4='BE0003856730 XBRU' /* ASC */
isin.5='BE0004608494 XBRU' /* BAB */
isin.6='DE0008600966 XAMS' /* BACFP */
isin.7='BE0004601424 XBRU' /* BAM */
isin.8='US0737301038 TNLA' /* BEAM */
isin.9='BE0020575115 XBRU' /* BELR */
isin.10='BE0003723377 XBRU' /* BELU */
isin.11='NL0000285278 XAMS' /* BEVER */
isin.12='US0970231058 TNLA' /* BOEI */
isin.13='BE0003825420 XBRU' /* CAMB */
isin.14='BE0004610516 XBRU' /* CATER */
isin.15='US1667641005 TNLA' /* CHTEX */
isin.16='BE0003304061 XBRU' /* CIM */
isin.17='BE0003811289 XBRU' /* COFP1 */
isin.18='BE0003813301 XBRU' /* COFP2 */
isin.19='BE0003624351 XBRU' /* DEFB */
isin.20='XS0138973010 XAMS' /* DEPCF */
isin.21='NL0009733351 XAMS' /* DICO */
isin.22='BE0004594355 XBRU' /* DOWB */
isin.23='NL0009901610 XBRU' /* ENVI */
isin.24='BE0004571122 XBRU' /* F */
isin.25='BE0003215143 XBRU' /* FLOB */
isin.26='US3448491049 TNLA' /* FOOTL */
isin.27='BE0003752665 XBRU' /* FOU */
isin.28='BE0004609500 XBRU' /* GEBC */
isin.29='NL0000440824 XAMS' /* GROHA */
isin.30='BE0004359916 XBRU' /* GT */
isin.31='NL0000358158 XAMS' /* HITT */
isin.32='ANN4327C1303 XAMS' /* HUNDP */
isin.33='NL0009391242 XAMS' /* HYDRA */
isin.34='US4592001014 TNLA' /* IBMA */
isin.35='US4595061015 TNLA' /* IFF */
isin.36='BE0003893139 XBRU' /* IMMOU */
isin.37='BE0004602430 XBRU' /* IMO */
isin.38='IL0010846314 XBRU' /* MEDV */
isin.39='BE0003731453 XBRU' /* MIKO */
isin.40='NL0000488153 XBRU' /* MOP */
isin.41='NL0000488161 XBRU' /* MOPF */
isin.42='BE0003602134 XBRU' /* MOUR */
isin.43='US5658491064 TNLA' /* MROIL */
isin.44='BE0003680916 XBRU' /* NEU */
isin.45='NL0009822014 XAMS' /* NSE */
isin.46='US6745991058 TNLA' /* OCPET */
isin.47='IL0010830391 XBRU' /* PAY */
isin.48='BE0003503118 XBRU' /* PCBB */
isin.49='US7134481081 TNLA' /* PEP */
isin.50='NL0000378669 XAMS' /* PORF */
isin.51='NL0009875483 XAMS' /* PREVA */
isin.52='US74928K2087 XAMS' /* RBST5 */
isin.53='US74928M2044 XAMS' /* RBST6 */
isin.54='US74928P2074 XAMS' /* RBST7 */
isin.55='BE0003654655 XBRU' /* SAB */
isin.56='BE0012378593 XBRU' /* SCHD */
isin.57='BE0003677888 XBRU' /* SER */
isin.58='BE0003545531 XBRU' /* SOLV */
isin.59='BE0003798155 XBRU' /* SPA */
isin.60='BE0003463685 XBRU' /* SUCR */
isin.61='BE0974263924 XBRU' /* TEXF */
isin.62='BE0003804219 XBRU' /* THI */
isin.63='NL0000388684 XAMS' /* UNCC7 */
isin.64='NL0000388742 XAMS' /* UNCP6 */
isin.65='NL0000388726 XAMS' /* UNCP7 */
isin.66='IL0010838311 XBRU' /* UNITB */
isin.67='US9078181081 TNLA' /* UNPA */
isin.68='NL0009082486 XAMS' /* VALUE */
isin.69='US92343V1044 TNLA' /* VERIZ */
isin.70='BE0003734481 XBRU' /* WEB */
isin.71='BE0003806230 XBRU' /* ZENT */

pd='--post-data="format=csv&layout=vertical&decimal_separator=.&'
pd=pd||'date_form=d/m/Y&op=Go&form_build_id=form-95ee71b0c76f01d'
pd=pd||'43248b53f98be27b2&form_id=nyx_data_center_download_form"'
ua='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US'
ua=ua||'; rv:1.5b)"'

IF Stream('AUCTION.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'AUCTION.TMP'
END
IF Stream('AUCTION2.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'AUCTION2.TMP'
END

DO i=1 TO isin.0
url='"https://europeanequities.nyx.com/nl/data/csv/order_boo'
url=url||'k?currentView=data&isin='||Word(isin.i,1)||'&mic='
url=url||Word(isin.i,2)||'"'
'@WGETSSL.EXE -q -OAUCTION.TMP' pd ua url
IF Stream('AUCTION.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL LineIn 'AUCTION.TMP'
regel=Strip(LineIn('AUCTION.TMP'))
IF Pos('-',regel,1)>0 THEN DO
PARSE VAR regel . '-' symbool
regel.0='Order book' Format(i,2)||'/'||isin.0 'of:'
regel.0=regel.0 Strip(symbool) '('||Word(isin.i,1)||')'
DO 5
CALL LineIn 'AUCTION.TMP'
END
laag=0
mkt=0
DO WHILE Lines('AUCTION.TMP')>0
regel=Strip(LineIn('AUCTION.TMP'))
IF Pos('"Total"',regel,1)=0 THEN DO
laag=laag+1
PARSE VAR regel '"' kooporders.laag '","' regel
PARSE VAR regel koopaantal.laag '","' regel
PARSE VAR regel biedprijs.laag '","' . '","' regel
PARSE VAR regel laatprijs.laag '","' regel
PARSE VAR regel verkoopaantal.laag '","' regel
PARSE VAR regel verkooporders.laag '"' .
IF laag=1 THEN DO
IF kooporders.laag>0 THEN DO
IF biedprijs.laag=0 THEN DO
mkt=1
biedprijs.laag='MKT'
END
END
IF verkooporders.laag>0 THEN DO
IF laatprijs.laag=0 THEN DO
mkt=1
laatprijs.laag='MKT'
END
END
END
IF kooporders.laag='' THEN DO
IF biedprijs.laag=0 THEN biedprijs.laag=''
END
IF verkooporders.laag='' THEN DO
IF laatprijs.laag=0 THEN laatprijs.laag=''
END
regel.laag=Right(kooporders.laag,4)
regel.laag=regel.laag Right(koopaantal.laag,9)
regel.laag=regel.laag Right(biedprijs.laag,9)
regel.laag=regel.laag Right(laatprijs.laag,9)
regel.laag=regel.laag Right(verkoopaantal.laag,9)
regel.laag=regel.laag Right(verkooporders.laag,4)
END
END
IF mkt=1 THEN DO
url='"https://europeanequities.nyx.com/nl/nyx_eu_l'
url=url||'istings/real-time/quote/'||Word(isin.i,1)
url=url||'-'||Word(isin.i,2)||'"'
'@WGETSSL.EXE' ua '-q -OAUCTION2.TMP' url
IF Stream('AUCTION2.TMP','C','QUERY EXISTS')<>'' THEN DO
omvang=Stream('AUCTION2.TMP','C','QUERY SIZE')
alles=CharIn('AUCTION2.TMP',1,omvang)
CALL CharOut 'AUCTION2.TMP'
CALL SysFileDelete 'AUCTION2.TMP'
PARSE VAR alles . 'id="datetimeLastvalue">' datum .
PARSE VAR alles . 'id="lastPriceint">' prefix '<' .
PARSE VAR alles . 'id="lastPricefract">' suffix '<' .
SAY regel.0
regel.0='Last:' Strip(prefix)||Strip(suffix)
regel.0=ChangeStr(',',regel.0,'.') '('||datum||')'
END
DO j=0 TO laag
SAY regel.j
END j
SAY
END
END
CALL LineOut 'AUCTION.TMP'
CALL SysFileDelete 'AUCTION.TMP'
END
END i

EXIT

Halt:

IF Stream('AUCTION.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'AUCTION.TMP'
END
IF Stream('AUCTION2.TMP','C','QUERY EXISTS')<>'' THEN DO
CALL SysFileDelete 'AUCTION2.TMP'
END

EXIT
A.D. Fundum
2012-04-25 17:20:10 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
Releasing Rexx apps, as-is, is no problem. The main issue
is that they'll be unclear
The final releasable pure Rexx-one. Actually the
ObjectRexx-interpreter is required, because of its Date()
implementation. Perhaps Rexx2Nrx also understands is, in the end
converting it to Java.

It's a lame attempt to calculate a ROACE (Return On Average Capital
Employed, and uses an input file, which may look like this. The
invested money is the initial margin of 4000, and we receive an option
premium of 82.70. It's not a zero-sum game, and if the option ends up
being worthless the ROI isn't 100% (of 82.70). The 5th column of the
output is a better, realistic indicator of the actual performance.


20120103 Invested 4000 Margin
20120103 Received 85-2.30 Open sell naked short call option - costs
20120129 Invested 500 Additional margin
20120217 Invested 0 Expiration out-of-the-money


Example of the output, using the input file above:

Date Capital Result AC ROACE
======== ========== ========= ============= =================
20120103 4082.70 82.70 4082.70 2.026%
20120104 4082.70 82.70 4082.70 2.026%
...
20120128 4082.70 82.70 4082.70 2.026%
20120129 4582.70 82.70 4101.22 2.016%
20120130 4582.70 82.70 4118.41 2.008%
...
20120216 4582.70 82.70 4293.81 1.926%
20120217 4582.70 82.70 4300.09 1.923%
--
/* ROACE.CMD */

/* Date I of R Mutatie Comment

20110915 Invested 100 - Purchase
20111001 Received 0.01 - Dividend
20111020 Invested 900 - Additional purchase
20111101 Received 1030-994 - (Virtual) ROI
20111108 Invested 0 - No change, i.e. the end */

CALL CharOut '','Which input file? '
PARSE PULL bron

regels=0

DO WHILE Lines(bron)>0
regels=regels+1
PARSE VALUE LineIn(bron) WITH datum.regels transactie.regels rest
PARSE VAR rest bedrag .
transactie.regels=Strip(Left(transactie.regels,1))
INTERPRET 'bedrag.regels='||bedrag
END

begin=Date('B',datum.1,'S')
einde=Date('B',datum.regels,'S')
dagen=einde-begin+1

saldo.=0
resultaat.=0
DO i=1 TO regels
IF transactie.i='R' THEN DO
DO j=Date('B',datum.i,'S') TO einde
resultaat.j=(resultaat.j)+(bedrag.i)
END j
transactie.i='I'
END
IF transactie.i='I' THEN DO
DO j=Date('B',datum.i,'S') TO einde
saldo.j=(saldo.j)+(bedrag.i)
END j
END
END

totaal=0
SAY 'Date Capital Result AC ROACE'
SAY '======== ========== ========= ============= ================='
DO i=begin TO einde
totaal=totaal+(saldo.i)
gewogen=totaal/(i-begin+1)
CALL CharOut '',Date('S',i,'B')||' '
CALL CharOut '',Format(saldo.i,7,2)||' '
CALL CharOut '',Format(resultaat.i,6,2)||' '
CALL CharOut '',Format(gewogen,10,2)||' '
SAY Format(((resultaat.i)/(gewogen))*100,12,3)||'%'
END i

EXIT
A.D. Fundum
2012-04-25 22:31:28 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
If I release an EXE without (c), I'll actually release the C
source. Why? Because you have to adjust it anyway. If
I haven't #defined my main index to be "AEX\0", the code
has to be adjusted. The main index determines what a
trading day is, other indici are included in code too,
and all apps assume my data format.
Raw data format (Metastock'ish): one sorted file per year (2011.PRN,
2012.PRN, ...), see e.g. days.zip @ Hobbes. If a line is missing
because the volume was 0, a line with a volume of 0 is added to a.o.
distinguish between lost data and a day without trade. An (unused)
header per file, and the first few lines to illustrate the file
format:


<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OI>
1NOMY,D,20120102,0.30,0.30,0.29,0.29,10700
1NOMY,D,20120103,0.29,0.29,0.29,0.29,300
1NOMY,D,20120104,0.29,0.29,0.29,0.29,0
1NOMY,D,20120107,0.29,0.31,0.271,0.30,1510


Below, without the matching HTML-producing, full name-adding Rexx
code, is RelVol.C. The first few lines of its output:


20120420 159
1NOMY 0.524625
3WPW 0.000000
AAA 0.316624
AALB 1.012646
ACCEL 0.658087


That's the date, number of lines. Next the 159 symbols and the 159
relative volumes. One may learn that a lot of internetting people are
"inventing" a relative volume (indicator), but this is a newspaper's
statistic. It can produce a quick market overview, it's a radar to
detect flying under the radar, if you've missed the news you may be
able to see which headlines you still should read, last week it may
have been a clue to look for e.g. a decreased or increased discount of
Shell shares A (0,93) and shares B (2,37).

But most often it's used socially. When I notice a high relative
volume of unfamiliar shares SBT, and I know you are a watcher of this
Swedish Bikini Team, it's a great way to start talking about it with
the SBT expert. Without really having to watch the SBT company myself,
I'll notice it just using this list and I'll maybe look at other
market-overviewing data too.

I've adjusted the code (a.o. length of lines, warpiing shouldn't be a
problem) and Anglofied some of the messages. By default all C code
should compile without compiler nor linker options.
--
/* Relative Volume (5:200 days) */

#define INCL_NOCOMMON
#define INCL_NOPMAPI

#include <float.h>
#include <os2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define INDEX1 "AEX\0"
#define INDEX2 "AMX\0"
#define INDEX3 "ASCX\0"
#define MAXDAGEN 265
#define MAXFONDSEN 256
#define PAD ""
#define REGELLENGTE 128

typedef struct _FONDSEN {char ticker[11];
double omzet5;
double omzet200;
USHORT dagen;} FONDSEN;

int main(int argc,char *argv[])
{
char bronnaam[CCHMAXPATH],csv[]=",",*d,doelnaam[CCHMAXPATH];
char *v,regel[REGELLENGTE],*t;
double dagomzet;
FILE *bron,*doel;
int i,j=0;
FONDSEN *fondsen;
USHORT gevonden=0,jaar,overslaan=3,tickers=0;
ULONG aex[MAXDAGEN*2],dag,datum,datum5,datum200;

if (argc!=2)
{
printf("\nCalculating relative volume 5:200 days\n\n");
printf(" Usage: RELVOL.EXE <YYYYMMDD>\n\n");
return 1;
}

datum=atol(argv[1]);
jaar=datum/10000;

if (NULL==(fondsen=calloc(MAXFONDSEN,sizeof(FONDSEN))))
{
printf("Error: cannot allocate enough memory\n");
return 1;
}

for (i=jaar-1;i<=jaar;i++)
{
sprintf(bronnaam,"%s%04u.PRN",PAD,i);
printf("Searching listed companies and trading days in ");
printf("file %s...\n",bronnaam);
if ((bron=fopen(bronnaam,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bronnaam);
free(fondsen);
return 1;
}
while (fgets(regel,REGELLENGTE,bron)!=NULL)
{
if (NULL!=(t=strtok(regel,csv)))
{
strtok(NULL,csv);
if (NULL!=(d=strtok(NULL,csv)))
{
dag=atol(d);
if (!strcmp(t,INDEX1))
{
if (dag<=datum)
{
aex[++j]=dag;
if (dag==datum)
gevonden=j;
}
}
if (i==jaar&&dag==datum)
strcpy(fondsen[++tickers].ticker,t);
}
}
}
fclose(bron);
}

if(!gevonden)
{
printf("\nError: trading day %8u not found in file ");
printf("%s\n",datum,bron);
free(fondsen);
return 1;
}

if (gevonden<200)
{
printf("\nError: %u instead of at least 200 trading ",gevonden);
printf(" days gevonden\n");
free(fondsen);
return 1;
}

printf("%u trading days found.\n",j);
printf("%u listed companies and indices found.\n",tickers);

datum5=aex[gevonden-4];
datum200=aex[gevonden-199];
printf("Found (200): %8u\n",datum200);
printf("Found ( 5): %8u\n",datum5);
printf("Found ( 1): %8u\n",datum);

for (i=jaar-1;i<=jaar;i++)
{
sprintf(bronnaam,"%s%04u.PRN",PAD,i);
printf("Searching volume data in file %s...\n",bronnaam);
if ((bron=fopen(bronnaam,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bronnaam);
free(fondsen);
return 1;
}
while (fgets(regel,REGELLENGTE,bron)!=NULL)
{
if (NULL!=(t=strtok(regel,csv)))
{
strtok(NULL,csv);
if (NULL!=(d=strtok(NULL,csv)))
{
dag=atol(d);
if (dag>=datum200&dag<=datum)
{
gevonden=0;
for (j=1;j<=tickers;j++)
if (!strcmp(fondsen[j].ticker,t))
gevonden=j;
if (gevonden)
fondsen[gevonden].dagen++;
strtok(NULL,csv);
strtok(NULL,csv);
strtok(NULL,csv);
strtok(NULL,csv);
if (NULL!=(v=strtok(NULL,csv)))
{
dagomzet=(double)atof(v);
if (dagomzet&&gevonden)
{
fondsen[gevonden].omzet200+=dagomzet;
if (dag>=datum5)
fondsen[gevonden].omzet5+=dagomzet;
}
}
}
}
}
}
fclose(bron);
}

sprintf(doelnaam,"RELVOL.PRN");
if ((doel=fopen(doelnaam,"w"))==NULL)
{
printf("Error: cannot create file %s\n",doelnaam);
free(fondsen);
return 1;
}

for (i=1;i<=tickers;i++)
{
if (fondsen[i].dagen!=200)
{
printf("Skipped, %3u trading days instead ",fondsen[i].dagen);
printf("of at least 200 found: %s\n",fondsen[i].ticker);
overslaan++;
}
}
if (tickers<=overslaan)
{
printf("Error: not enough data available\n");
free(fondsen);
return 1;
}

fprintf(doel,"%8u %u\n",datum,tickers-overslaan);

for (i=1;i<=tickers;i++)
{
if (!fondsen[i].omzet200)
fondsen[i].omzet200=1;
if (fondsen[i].dagen==200&&strcmp(fondsen[i].ticker,INDEX1))
{
if (strcmp(fondsen[i].ticker,INDEX2))
{
if (strcmp(fondsen[i].ticker,INDEX3))
fprintf(doel,
"%-10s %13.6f\n",
fondsen[i].ticker,
(float)(fondsen[i].omzet5*(double)40/fondsen[i].omzet200));
}
}
}
fclose(doel);

printf("File %s created.\n",doelnaam);

free(fondsen);

return 0;
}
A.D. Fundum
2012-04-26 08:06:26 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
If I release an EXE without (c), I'll actually release the C
source.

Once answering yet another is-there-a-free-website-which-...-question.

Required: gDiagramm.EXE, the latest version is recommended. And at
least 2 Metastock'ish files, e.g. 2011.PRN and 2012,PRN. Not tested,
but it should already work with this data:


2011.PRN:

<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OI>
SBT,D,20111231,0.29,0.29,0.29,0.29,1


2012.PRN:

<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OI>
SBT,D,20120101,0.295,0.295,0.295,0.295,1
SBT,D,20120102,0.28,0.28,0.28,0.28,1




The line with "if (verschil[i]>0)" avoids a gDiagramm bug. ISTR it
won't draw a graph with a flat data line (and it has to draw a line).
As usual, the HTML file it produces can be viewed just fine with
Netscape.

Possible improvements, other than an UI, and so on: the date argument
isn't checked, albeit that's not that important. More lines, more
settings. IRL an improvement is fixing the gDiagramm flat line-bug
(some source code available, but AFAICT it doesn't create the original
EXE out-of-the-box). And even more important a DLL version of
gDiagramm, so the EXE isn't loaded twice per processed symbol.
gDiagramm itself is pretty fast indeed, certainly the newer versions,
but this is by far the slowest market overviewing-releated app I've
got.

BTW, "BIN" refers to the directory where GDIAGRAMM.EXE can be found,
to speed things up. It has nothing to do with Unix at all, I'm not
using Unix nor any Unix file or directory structure.
--
/* Graphs.C - Creates small and large graphs */

#define INCL_DOSFILEMGR
#define INCL_NOCOMMON
#define INCL_WINWINDOWMGR

#pragma import(WinQueryDesktopWorkArea,,"PMMERGE",5469)

#include <io.h>
#include <os2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

BOOL APIENTRY WinQueryDesktopWorkArea(HWND,PRECTL);

#define BIN ""
#define MAXFONDSEN 256
#define PAD ""
#define REGELLENGTE 128

typedef struct _TICK {char ticker[11];ULONG vanaf;} TICK;

int main(int argc, char *argv[])
{
char buffer[CCHMAXPATH],buffer2[15],csv[]=",",*d,*decimalen;
char kopieregel[32],posities,regel[REGELLENGTE],*s,*t;
char verschil[MAXFONDSEN],vorige[32],wissel[11];
FILE *bron,*doel;
long orgdatum,schermbreedte=1024,schermhoogte=768;
RECTL pD;
short i,j,jaar;
ULONG dag,hulp,vanaf;
USHORT breedte,einde,hoogte,teller=0,teller2;
TICK fonds[MAXFONDSEN];

if (argc!=2)
{
printf("Small and large graphs\n\n");
printf(" Usage: GRAPHS.EXE <YYYYMMDD>\n");
return 1;
}

orgdatum=atol(argv[1]);

if (orgdatum<20070101||orgdatum>20971231)
{
printf("Small and large graphs\n\n");
printf(" Usage: GRAPHS.EXE <YYYYMMDD>\n");
return 0;
}

if (WinQueryDesktopWorkArea(HWND_DESKTOP,&pD))
{
schermbreedte=pD.xRight-pD.xLeft;
schermhoogte=pD.yTop-pD.yBottom;
}
if ((schermbreedte/4)*3>schermhoogte)
schermbreedte=(schermhoogte/3)*4;
else if ((schermbreedte/4)*3<schermhoogte)
schermhoogte=(schermbreedte/4)*3;

vanaf=orgdatum-10000;
einde=orgdatum/10000;
for (i=0;i<MAXFONDSEN;i++)
fonds[i].vanaf=99999999;

for (jaar=einde-1;jaar<=einde;jaar++)
{
sprintf(buffer,"%s%04u.PRN",PAD,jaar);
printf("Busy processing file %s...\n",buffer);
if ((bron=fopen(buffer,"r"))==NULL)
{
printf("Error: cannot open file %s\n",buffer);
return 0;
}
fgets(regel,REGELLENGTE,bron);
while (fgets(regel,REGELLENGTE,bron)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
dag=atol(d);
if (dag>vanaf&&dag<=orgdatum)
{
strtok(NULL,csv);
strtok(NULL,csv);
strtok(NULL,csv);
if ((s=strtok(NULL,csv))!=NULL)
{
sprintf(buffer2,"%s.TMP",t);
if ((access(buffer2,0))==-1)
sprintf(fonds[++teller].ticker,"%s",t);

for (i=1;i<=teller;i++)
{
if (!strcmp(fonds[i].ticker,t))
{
fonds[i].vanaf=min(dag,fonds[i].vanaf);
i=teller;
}
}
if ((doel=fopen(buffer2,"a"))==NULL)
{
printf("Error: cannot open file %s\n",buffer2);
return 0;
}
fprintf(doel,"%s\n",s);
fclose(doel);
}
}
}
}
}
fclose(bron);
}

if (!teller)
return 0;
printf("Creating 2x %u graph specifications...\n",teller);
breedte=(schermbreedte-60)/3;
hoogte=((breedte/1.5)+0.5);
sprintf(csv,".");

if (teller>1)
{
i=1;
while (i<teller)
{
if (strcmp(fonds[i].ticker,fonds[i+1].ticker)>0)
{
sprintf(wissel,"%s",fonds[i].ticker);
hulp=fonds[i].vanaf;
sprintf(fonds[i].ticker,"%s",fonds[i+1].ticker);
fonds[i].vanaf=fonds[i+1].vanaf;
sprintf(fonds[i+1].ticker,"%s",wissel);
fonds[i+1].vanaf=hulp;
i=0;
}
i++;
}
}

for (i=1;i<=teller;i++)
{
verschil[i]=0;
sprintf(buffer,"%s.TMP",fonds[i].ticker);
if ((bron=fopen(buffer,"r"))==NULL)
{
printf("Error: cannot open file %s\n",buffer);
return 0;
}
posities=0;
teller2=0;
while (fgets(regel,32,bron)!=NULL)
{
sprintf(kopieregel,"%s",regel);
teller2++;
if (strtok(regel,csv)!=NULL)
{
if ((decimalen=strtok(NULL,csv))!=NULL)
{
if (strlen(decimalen)-1>posities)
posities=strlen(decimalen)-1;
if (teller2>1)
{
if (strcmp(kopieregel,vorige))
verschil[i]=1;
}
sprintf(vorige,"%s",kopieregel);
}
}
}
fclose(bron);
if (verschil[i]>0)
{
sprintf(buffer,"%s.FMT",fonds[i].ticker);
if ((doel=fopen(buffer,"w"))==NULL)
{
printf("Error: cannot create file %s\n",buffer);
return 0;
}
fprintf(doel,"form.Format=%%lf\n");
fprintf(doel,"form.DimX=%u\n",breedte);
fprintf(doel,"form.DimY=%u\n",hoogte);
fprintf(doel,"form.FileOut=%s.PNG\n",fonds[i].ticker);
fprintf(doel,"form.LegendUse=0\n");
fprintf(doel,
"form.Title=Year graph %s since %u\n",
fonds[i].ticker,
fonds[i].vanaf);
fprintf(doel,"form.BackColor=0xd9d9d9\n");
fprintf(doel,"form.FormatX=%%.0d\n");
fprintf(doel,"form.FormatY=%%.%df\n",posities);
fprintf(doel,"field0.use=1\n");
fprintf(doel,"field0.color=0x1c1c1c\n");
fclose(doel);
sprintf(buffer,"%slarge.FMT",fonds[i].ticker);
if ((doel=fopen(buffer,"w"))==NULL)
{
printf("Error: cannot create file %s\n",buffer);
return 0;
}
fprintf(doel,"form.Format=%%lf\n");
fprintf(doel,"form.DimX=%u\n",schermbreedte-24);
fprintf(doel,"form.DimY=%u\n",schermhoogte-184);
fprintf(doel,"form.FileOut=%slarge.PNG\n",fonds[i].ticker);
fprintf(doel,"form.LegendUse=0\n");
fprintf(doel,
"form.Title=Year graph %s since %u\n",
fonds[i].ticker,
fonds[i].vanaf);
fprintf(doel,"form.BackColor=0xd9d9d9\n");
fprintf(doel,"form.FormatX=%%.0d\n");
fprintf(doel,"form.FormatY=%%.%df\n",posities);
fprintf(doel,"field0.use=1\n");
fprintf(doel,"field0.color=0x1c1c1c\n");
fclose(doel);
}
}

printf("Creating 2x %u graphs...\n",teller);
for (i=1;i<=teller;i++)
{
if (verschil[i]>0)
{
printf("Busy with: %-10s\r",fonds[i].ticker);
fflush(stdout);
sprintf(buffer,
"@%sgDiagramm.EXE %s.TMP %s.FMT > NUL",
BIN,
fonds[i].ticker,
fonds[i].ticker);
system(buffer);
sprintf(buffer,
"@%sgDiagramm.EXE %s.TMP %slarge.FMT > NUL",
BIN,
fonds[i].ticker,
fonds[i].ticker);
system(buffer);
sprintf(buffer,"%s.FMT",fonds[i].ticker);
DosForceDelete(buffer);
sprintf(buffer,"%slarge.FMT",fonds[i].ticker);
DosForceDelete(buffer);
sprintf(buffer,"%s.TMP",fonds[i].ticker);
}
else
{
sprintf(buffer,"%s.TMP",fonds[i].ticker);
if (i<teller)
{
for (j=i;j<teller;j++)
{
sprintf(fonds[j].ticker,"%s",fonds[j+1].ticker);
verschil[j]=verschil[j+1];
}
i--;
}
teller--;
}
DosForceDelete(buffer);
}

printf("Sorting\n");
i=1;
while (i<teller)
{
j=i+1;
if (strcmp(fonds[i].ticker,fonds[j].ticker)>0)
{
sprintf(regel,"%s",fonds[j].ticker);
strcpy(fonds[j].ticker,fonds[i].ticker);
strcpy(fonds[i].ticker,regel);
i=0;
}
i++;
}

printf("\nCreating HTML...\n");
sprintf(buffer,"yeargraphs.html");
if (!access(buffer,0))
DosForceDelete(buffer);
if ((doel=fopen(buffer,"w"))==NULL)
{
printf("Error: cannot create file %s\n",buffer);
return 0;
}
fprintf(doel,"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Tr");
fprintf(doel,"ansitional//EN\" \"http://www.w3.org/TR/html4/lo");
fprintf(doel,"ose.dtd\">\n");
fprintf(doel,"<HTML><HEAD><TITLE>Year graphs</TITLE></HEAD>\n");
fprintf(doel,"<BODY bgcolor=\"#0070bf\" text=\"#ffffff\" link=");
fprintf(doel,"\"#ffff00\" vlink=\"#00ffff\" alink=\"#ffc0cb\">\n");
fprintf(doel,"<BR><CENTER><H2>Year graph closing prices</H2>");
fprintf(doel,"</CENTER>\n");
fprintf(doel,"<HR align=\"center\" width=\"80%%\"><BR>\n");
fprintf(doel,"<TABLE border=\"0\" align=\"center\" bgcolor=\"#");
fprintf(doel,"0090bf\">\n");
for (i=1;i<=teller;i+=3)
{
fprintf(doel,"<TR>");
for (j=i;j<=i+2;j++)
{
if (j<=teller)
{
fprintf(doel,"<TD><img src=\"%s.PNG\" ",fonds[j].ticker);
fprintf(doel,"alt=\"%s\" ",fonds[j].ticker);
fprintf(doel,"align=\"center\" border=\"0\" height=");
fprintf(doel,"\"%u\" width=\"%u\"></TD>",hoogte,breedte);
}
}
fprintf(doel,"</TR>\n<TR>");
for (j=i;j<=i+2;j++)
{
if (j<=teller)
{
fprintf(doel,"<TD align=\"center\"><A href=\"");
fprintf(doel,"%slarge.PNG\">",fonds[j].ticker);
fprintf(doel,"%s</A></TD>",fonds[j].ticker);
}
}
fprintf(doel,"</TR>\n");
}
fprintf(doel,"</TABLE>\n");
fprintf(doel,"<HR align=\"center\" width=\"80%%\"><BR clear=\"al");
fprintf(doel,"l\"></BODY></HTML>\n");
fclose(doel);

return 0;
}
A.D. Fundum
2012-04-26 13:59:04 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
If I release an EXE without (c), I'll actually release the C
source.

The average intraday volatility, i.e. high/low*100-100 (appr. the
missing, magic X in the well-known B&S formula). Yet another way to
obtain a quick market overview.

"Cheap" pump&dump crap is easy to detect, but another use is being
aware of the fact that the average intraday volatility of e.g. SBT is
2.53%. If today's high is $15.15 and the current price is $14.99, it's
not unlikely that you'ld be able to buy SBT at $14.78. That'll be
nomal. Of course it doesn't work that way. The trading range may as
well become 14.99-15.36, but it's important to know what's a normal
range. It's often the easiest way to upgrade your ROI with 1% (still
less than 50% of the 2.53%), at least compared to buying like there's
no tomorrow and thinking that $14.99 is a bargain. It isn't. You don't
even have to be patient, the example of 2.53% is a daily average. I.e.
a perfect trader can make a 2.53% a day, try to avoid paying for that.

Example of its output, again using a Metastock'ish PRN file:


20120102 20120425 165
1NOMY 1.00
3WPW 17.64
AAA 1.26
AALB 2.49
ACCEL 2.27
...
PHARM 5.41
PHIA 2.34
PNL 3.80
PORF 0.19
...


A possible improvement is an UI which can combine it with other data.
The 17.64% of 3WPW, AFAIK a worthless warrant, is explained by rare
records like:

3WPW,D,20120223,0.005,0.011,0.001,0.011,320050
--
/* Vola.C - Average intraday volatility */

#define INCL_NOCOMMON
#define INCL_NOPMAPI

#include <float.h>
#include <math.h>
#include <os2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAXFONDSEN 256
#define PAD ""
#define REGELLENGTE 128

typedef struct _TICK {char ticker[11];
float uitslag;
USHORT handelsdagen;} TICK;

int main(int argc,char *argv[])
{
char bron[CCHMAXPATH],csv[]=",",*d,*h,*l,regel[REGELLENGTE],*t;
FILE *file;
TICK fonds[MAXFONDSEN];
USHORT gevonden,i,j,tickers=0;

if (argc!=3)
{
printf("\nAverage intraday volatility\n\n");
printf(" Usage: VOLA.EXE <YYYYMMDD> <YYYYMMDD>\n\n");
return 1;
}

if (atoi(argv[1])>atoi(argv[2]))
{
printf("\nError: date %s is not older than ",argv[1]);
printf("date %s\n",argv[2]);
return 1;
}

for (i=0;i<MAXFONDSEN;i++)
{
fonds[i].uitslag=0;
fonds[i].handelsdagen=0;
}

for (i=atoi(argv[1])/10000;i<=atoi(argv[2])/10000;i++)
{
sprintf(bron,"%s%04u.PRN",PAD,i);
if ((file=fopen(bron,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bron);
return 1;
}
printf("Searching data in file %s...\n",bron);
fgets(regel,REGELLENGTE,file);
while (fgets(regel,REGELLENGTE,file)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
strtok(NULL,csv);
if ((h=strtok(NULL,csv))!=NULL)
{
if ((l=strtok(NULL,csv))!=NULL)
{
if (!strcmp(d,argv[1]))
sprintf(fonds[++tickers].ticker,"%s",t);
if ((strcmp(d,argv[1])>=0)&&(strcmp(d,argv[2])<=0))
{
gevonden=0;
for (j=1;j<=tickers;j++)
if (!strcmp(fonds[j].ticker,t))
gevonden=j;
if (gevonden)
{
fonds[gevonden].uitslag+=((atof(h)*100)/atof(l))-100;
fonds[gevonden].handelsdagen++;
}
}
}
}
}
}
}
fclose(file);
}

gevonden=0;
for (i=1;i<=tickers;i++)
if (!fonds[i].handelsdagen)
gevonden++;

printf("Data found for %u symbols.\n",tickers);
printf("Of which not found at %s: %u\n",argv[2],gevonden);

if ((file=fopen("VOLA.PRN","w"))==NULL)
{
printf("Error: cannot create file VOLA.PRN\n");
return 1;
}
fprintf(file,"%s %s %u\n",argv[1],argv[2],tickers-gevonden);
for (i=1;i<=tickers;i++)
{
if (fonds[i].handelsdagen)
fprintf(file,
"%-15s %9.2f\n",
fonds[i].ticker,
fonds[i].uitslag/fonds[i].handelsdagen);
}
fclose(file);

printf("File VOLA.PRN created.\n");

return 0;
}
A.D. Fundum
2012-04-26 13:59:03 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
BTW, this is an example. If you somehow don't understand the
essence of C source and I don't write a manual, it'll be as useful
as the so-called book The Da Dinvi Code, representing Stephen
http://home.uni-one.nl/m1/liquiditeitsindices.html
(link may now actually work for a while)

And here it is. I made this one up. Again the goal is to provide a
quick market overview. The third column of the output file typically
suprises a daytrader. Because they tend to prefer certain listed
companies, and won't notice there's more to love and there's often
more to add to their watchlists. The second column of the output has
an use for almost everybody, for example because it shows how easy it
ought to be to buy or sell such a share.

I should add that the calculated index isn't fine-tuned, for one
because there'll be more meaningless 0.00 then. YMMV, but an index of
about 50 indicates it's likely an order will be executed rather soon
and you don't have to be that patient. The fourth colum of the output
file can be used as a secondary sorting key, albeit small differences
are also meaningless.

Known "error": the score of newly listed companies often is too high
due to the implied lower reductions. Earlier I've added the
traded/listed since-date to this file, to be able to add it to the
final (HTML) output, but IRL I'll recognize 'em anyway.

BTW, I haven't posted the HTML postprocessor because that adds another
data file format, like the full name belonging to a symbol. This one
just requires the Metastock'ish files again. The main stock exchange
index, INDEX1, is used again to determine the trading days. It
supports up to 3 indici (i.e. records without traded volume) in the
data file, but that can be changed.
--
/* Liq.C - Calculating liquidity indici */

#define INCL_NOCOMMON
#define INCL_NOPMAPI

#include <float.h>
#include <os2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define INDEX1 "AEX\0"
#define INDEX2 "AMX\0"
#define INDEX3 "ASCX\0"
#define MAXDAGEN 265
#define MAXFONDSEN 256
#define PAD ""
#define REGELLENGTE 128

typedef struct _TICK {char ticker[11];
float traddays;
float hols;
float hulpomzet;
float index;
float index2;
float minmarge;
float minmarge2;
float minomzet;
float nulomzet;
float over25;} TICK;

int main(int argc,char *argv[])
{
char bron[CCHMAXPATH],csv[]=",",*d,*h,*l,*o;
char regel[REGELLENGTE],*s,*t,*v;
FILE *file;
float dagomzet;
TICK fonds[MAXFONDSEN];
ULONG aex[MAXDAGEN*2],dag,datum,vanaf,vanaf2=99999999;
USHORT found=0,i,j,jaar,jaar2,teller=0,tickers=0;

if (argc!=2)
{
printf("\nCalculating liquidity indici\n\n");
printf(" Usage: LIQ.EXE <YYYYMMDD>\n\n");
return 1;
}

datum=atol(argv[1]);
jaar2=datum/10000;
vanaf=datum-10000;

for (i=0;i<MAXFONDSEN;i++)
{
fonds[i].traddays=0;
fonds[i].hols=0;
fonds[i].hulpomzet=0;
fonds[i].index=100;
fonds[i].index2=100;
fonds[i].minmarge=0;
fonds[i].minmarge2=0;
fonds[i].minomzet=0;
fonds[i].nulomzet=0;
fonds[i].over25=0;
}

for (jaar=jaar2-1;jaar<=jaar2;jaar++)
{
sprintf(bron,"%s%04u.PRN",PAD,jaar);
if ((file=fopen(bron,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bron);
return 1;
}
printf("Searching symbols and trading days in file %s...\n",bron);
fgets(regel,REGELLENGTE,file);
while (fgets(regel,REGELLENGTE,file)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
dag=atol(d);
if (!strcmp(t,INDEX1))
{
if (dag<=datum&&dag>vanaf)
{
vanaf2=min(dag,vanaf2);
aex[++teller]=dag;
if (dag==datum)
found=teller;
}
}
if (dag==datum)
strcpy(fonds[++tickers].ticker,t);
}
}
}
fclose(file);
}

if (!found)
{
printf("\nError: trading day %u not found in ",datum);
printf("file %s.\n",bron);
return 1;
}

printf("%u trading days found.\n",teller);
printf("%u symbols found.\n",tickers);

printf("Found (year): %08u\n",vanaf2);
printf("Found ( 1): %08u\n",datum);

for (jaar=jaar2-1;jaar<=jaar2;jaar++)
{
sprintf(bron,"%s%04u.PRN",PAD,jaar);
if ((file=fopen(bron,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bron);
return 1;
}
printf("Processing file %s to calculate estimated turnover",bron);
printf("and trading days...\n");
fgets(regel,REGELLENGTE,file);
while (fgets(regel,REGELLENGTE,file)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
dag=atol(d);
if ((o=strtok(NULL,csv))!=NULL&&(h=strtok(NULL,csv))!=NULL)
{
if ((l=strtok(NULL,csv))!=NULL&&(s=strtok(NULL,csv))!=NULL)
{
if ((v=strtok(NULL,csv))!=NULL&&(dag<=datum&&dag>=vanaf2))
{
found=0;
for (j=1;j<=tickers;j++)
{
if (!strcmp(fonds[j].ticker,t))
found=j;
}
dagomzet=((atof(o)+atof(h)+atof(l)+atof(s))/4)*atof(v);
if (found)
{
fonds[found].traddays++;
fonds[found].hulpomzet+=dagomzet;
}
}
}
}
}
}
}
fclose(file);
}

for (jaar=jaar2-1;jaar<=jaar2;jaar++)
{
sprintf(bron,"%s%04u.PRN",PAD,jaar);
if ((file=fopen(bron,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bron);
return 1;
}
printf("Reading file %s to calculate liquidity indici...\n",bron);
fgets(regel,REGELLENGTE,file);
while (fgets(regel,REGELLENGTE,file)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
dag=atol(d);
if ((o=strtok(NULL,csv))!=NULL&&(h=strtok(NULL,csv))!=NULL)
{
if ((l=strtok(NULL,csv))!=NULL&&(s=strtok(NULL,csv))!=NULL)
{
if ((v=strtok(NULL,csv))!=NULL&&dag<=datum&&dag>=vanaf2)
{
found=0;
for (j=1;j<=tickers;j++)
{
if (!strcmp(fonds[j].ticker,t))
found=j;
}
dagomzet=((atof(o)+atof(h)+atof(l)+atof(s))/4)*atof(v);
if (found)
{
if ((atof(h)==atof(l))&&atof(v))
fonds[found].hols++;
if (!atof(v))
fonds[found].nulomzet++;
if (dagomzet>=5000)
fonds[found].minomzet++;
if ((dagomzet*4)>=((fonds[found].hulpomzet)/(fonds[found].traddays)))
fonds[found].over25++;
if ((atof(h)/atof(l))>=1.004)
fonds[found].minmarge++;
if ((atof(h)/atof(l))>=1.02)
fonds[found].minmarge2++;
}
}
}
}
}
}
}
fclose(file);
}

if ((file=fopen("LIQ.PRN","w"))==NULL)
{
printf("Error: cannot create file LIQ.PRN\n");
return 1;
}
fprintf(file,"%08u %08u\n",vanaf2,datum);
for (i=1;i<=tickers;i++)
{
if (strcmp(fonds[i].ticker,INDEX1)&&strcmp(fonds[i].ticker,INDEX2))
{
if (strcmp(fonds[i].ticker,INDEX3))
{
fonds[i].hols=fonds[i].traddays-fonds[i].hols;
fonds[i].nulomzet=fonds[i].traddays-fonds[i].nulomzet;
fonds[i].index*=(float)(fonds[i].hols/fonds[i].traddays);
fonds[i].index*=(float)(fonds[i].minmarge/fonds[i].traddays);
fonds[i].index*=(float)(fonds[i].minomzet/fonds[i].traddays);
fonds[i].index*=(float)(fonds[i].nulomzet/fonds[i].traddays);
fonds[i].index*=(float)(fonds[i].over25/fonds[i].traddays);
fonds[i].index2*=(float)(fonds[i].hols/fonds[i].traddays);
fonds[i].index2*=(float)(fonds[i].minmarge2/fonds[i].traddays);
fonds[i].index2*=(float)(fonds[i].minomzet/fonds[i].traddays);
fonds[i].index2*=(float)(fonds[i].nulomzet/fonds[i].traddays);
fonds[i].index2*=(float)(fonds[i].over25/fonds[i].traddays);
fprintf(file,
"%-15s %12.8f %12.8f %18.2f\n",
fonds[i].ticker,
fonds[i].index,
fonds[i].index2,
fonds[i].hulpomzet);
}
}
}
fclose(file);

printf("File LIQ.PRN created.\n");

return 0;
}
A.D. Fundum
2012-04-26 13:59:04 UTC
Permalink
Post by A.D. Fundum
Post by Will Honea
http://moneytide.com/hans/index.asp
If I release an EXE without (c), I'll actually release the C
source.

And probably the last one in the C department, a basic
gainer/loser/unchanged report.

Output example (GALOUN.EXE 20120102 20120425):


20120102 20120425 165
...
ALTMC 9.6500 14.5000 50.2591 4.8500
AMG 7.8000 7.8200 0.2564 0.0200
AMIGO 3.9600 3.8700 -2.2727 0.0900
AMT 0.3500 0.0450 -87.1429 0.3050
...


Apparently AMG gained 0.2564%. The fifth column, abs(difference), was
added to be able to reject gains of 100% or $0.0001, albeit I'm not
using that data field. An example of a data file 2012.PRN, with
deleted records between both days, could heve been:


<TICKER>,<PER>,<DTYYYYMMDD>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OI>
AMG,D,20120102,7.693,7.83,7.501,7.80,214483
AMG,D,20120425,7.50,7.90,7.50,7.82,164710


I've got more C files, but those are less generic. For example an app
based on turnover, which can be ued to write an educated financial
story about the exchange-related relevancy of a small company you've
never really heard of before. As stated earlier, by far the most fun
facts and myth busting (Rexx) apps were already deleted. Often because
reusing those was unlikely. Those deleted Rexx apps could have been be
added to some "toolbox", but as such those apps were hardly
stand-alone apps. As if we'ld release a creativly used
spreadsheet-function, and we've forgotten what the matching original
question was.


--

/* GaLoUn.C - Gained/lost/unchanged */

#define INCL_NOCOMMON
#define INCL_NOPMAPI

#include <float.h>
#include <math.h>
#include <os2.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAXFONDSEN 256
#define PAD ""
#define REGELLENGTE 128

typedef struct _FONDSEN {char ticker[11];
float koers1;
float koers2;} FONDSEN;

int main(int argc,char *argv[])
{
char bronnaam[CCHMAXPATH],csv[]=",",*d,doelnaam[CCHMAXPATH];
char regel[REGELLENGTE],*s,*t;
FILE *bron,*doel;
FONDSEN *fondsen;
USHORT gevonden,i,j,tickers=0;

if (argc!=3)
{
printf("\nGained/lost/unchanged\n\n");
printf(" Usage: GALOUN.EXE <YYYYMMDD> <YYYYMMDD>\n\n");
return 1;
}

if (atol(argv[1])>=atol(argv[2]))
{
printf("\nError: date %s is not older than ",argv[1]);
printf("date %s\n",argv[2]);
return 1;
}

if (NULL==(fondsen=calloc(MAXFONDSEN,sizeof(FONDSEN))))
{
printf("Error: cannot allocate enough memory\n");
return 1;
}

for (i=atoi(argv[1])/10000;i<=atoi(argv[2])/10000;i++)
{
sprintf(bronnaam,"%s%04u.PRN",PAD,i);
printf("Searching quotes in file %s...\n",bronnaam);
if ((bron=fopen(bronnaam,"r"))==NULL)
{
printf("Error: cannot open file %s\n",bronnaam);
free(fondsen);
return 1;
}
fgets(regel,REGELLENGTE,bron);
while (fgets(regel,REGELLENGTE,bron)!=NULL)
{
if ((t=strtok(regel,csv))!=NULL)
{
strtok(NULL,csv);
if ((d=strtok(NULL,csv))!=NULL)
{
strtok(NULL,csv);
strtok(NULL,csv);
strtok(NULL,csv);
if ((s=strtok(NULL,csv))!=NULL)
{
if (!strcmp(argv[1],d))
{
sprintf(fondsen[++tickers].ticker,"%s",t);
fondsen[tickers].koers1=atof(s);
}
if (!strcmp(argv[2],d))
{
gevonden=0;
for (j=1;j<=tickers;j++)
{
if (!strcmp(fondsen[j].ticker,t))
gevonden=j;
}
if (gevonden)
fondsen[gevonden].koers2=atof(s);
}
}
}
}
}
fclose(bron);
}

gevonden=0;
for (i=1;i<=tickers;i++)
if (!fondsen[i].koers2)
gevonden++;

printf("Data found of %u symbols.\n",tickers);
printf("Of which not found at date %s: %u\n",argv[2],gevonden);

sprintf(doelnaam,"GALOUN.PRN");
if ((doel=fopen(doelnaam,"w"))==NULL)
{
printf("Error: cannot create file %s\n",doelnaam);
free(fondsen);
return 1;
}
fprintf(doel,"%s %s %u\n",argv[1],argv[2],tickers-gevonden);
for (i=1;i<=tickers;i++)
{
if (fondsen[i].koers2)
fprintf(doel,
"%-10s %14.4lf %14.4lf %14.4lf %14.4lf\n",
fondsen[i].ticker,
fondsen[i].koers1,
fondsen[i].koers2,
((fondsen[i].koers2*100)/fondsen[i].koers1)-100,
fabs(fondsen[i].koers2-fondsen[i].koers1));
}
fclose(doel);

printf("File %s created.\n",doelnaam);

free(fondsen);

return 0;
}
A.D. Fundum
2012-06-22 18:33:36 UTC
Permalink
Post by Will Honea
http://moneytide.com/hans/index.asp
PCX2PRN is another example
PCX2PRN 1.3.0 is released. If you've heard that zinc dropped 35% last
year, all you can find to play with is a free zinc price chart (like
http://www.lme.com/zinc_graphs.asp), and the quality of this line
graph isn't poor, then the number of steps to convert this line graph
to numbers now is reduced to just 4: download the graph, save it as a
TGA file with upto 256 colors, find out the color of the line, and
create the text file.

It's not in the money-section of Hobbes, albeit it's unlikely that one
wants to convert digital photos of a wedding to depressing numbers:

http://hobbes.nmsu.edu/download/pub/incoming/pcx2prn130.zip
http://hobbes.nmsu.edu/download/pub/os2/apps/graphics/convert/pcx2prn1
30.zip


--
A.D. Fundum
2012-04-22 12:50:36 UTC
Permalink
Post by A.D. Fundum
auction
Here's an example of the mentioned quick&dirty auction hunting app,
looking for a discount related to sellers without buyers:

Order book 27/36 of: RBS CAP FUND TRVII (RBST7, US74928P2074)
Last price: 12.00
1 500 10.10 At Best 1 1
1 500 10.09 13.69 5 1
0 0 - 13.70 33 1

Remarks: it's translated (no problem) and Anglified (. - ,). It
doesn't look at ammounts, this is a EUR 10.11 business opportunity
involving one share of the Royal Bank of Scotland Capital Funding
Trust VII. There's no link with a portfolio, The (36, at the moment )
stocks are hard-coded in the app, I haven't added over about 700
bonds. It assumes local stock exchange rules, which aren't embedded.

Of course the underlying idea, using rules to the max and assuming we
potentionally want to buy those shares at the right price/value, is to
buy the share at 90% of 12.00 (the allowed maximum discount probably
is 10% per auction, yet another specific condition and rule). You
won't be rich quick, but it's also fun. If it works, depending on the
final situation at the moment of the auction, one can buy it for
10.80. The newspaper (and websites, nowadays) will list the -10%,
which on its own attracts buyers at +10%. So we buy at 10.80, and sell
it soon at 11.88 (+10%) or at the normal price or value of 12.00.

It works (with a current NYSE website, that is). But if you would
execute it and see no output (no market orders at the moment), it's
unclear. If there's output, the output is unclear. Not everybody can
read an order book, and those who (think they) can often don't
understand the opportunity. The app itself is quite unique and has an
use, but there's a lot to explain and a lot of people like to pretend
they don't care about $5,000 investments.

The app can be improved by querying the currect stocks trading using
an auction, and so on, but THIS is the app I've written. It just
conditionally dumps order books, and assumes the user both understands
and appriciates the text mode output, if there's output.


--
A.D. Fundum
2012-04-25 16:04:30 UTC
Permalink
Post by A.D. Fundum
I'm out of (this) business for an unknown period of time,
because of a "changed" (and broken, and ...) NYSE
website.
Sigh...

"We've done more than just improve our look. We've completely
redesigned our websites from the ground up with expanded resources,
streamlined navigation and faster access to information, data and
unique industry perspective."

Thanks Goodness they're not finished yet, because there's no sign of
"faster access to information". So far the "unique industry
perspective" comes down to slower access to no information.


--
A.D. Fundum
2015-02-08 21:41:32 UTC
Permalink
Post by A.D. Fundum
I've a collection of small apps
Such a collection is uploaded to Hobbes, FWIW:

http://hobbes.nmsu.edu/download/pub/incoming/stockquotedatareports100.
zip


--

Loading...