Data Analysis tools

New to FreeBASIC? Post your questions here.
Post Reply
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Data Analysis tools

Post by dasyar »

I thought I would open this topic up for discussion, not even sure if freeBasic would be the place to start for this. I am also not even sure as to where this conversion should start, maybe the simplest form of data analysis, a requested form of a grouping of data points?

How could you use freeBasic commands to open file, a .csv file, and get a specified grouping of some data. As an example, you have a column of dates that is tied to a specific value, and you want a grouping of some values for a particular date. Not sure if this is the best place to start.

All commentary is welcome, even if it is skeptical.

Thanks
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Data Analysis tools

Post by St_W »

Of course this depends on the type of data analysis you want to do, but I would tend to say that FreeBasic is not the ideal language for that application area. I'd suggest to use e.g. R: https://www.r-project.org/
In contrast to FB this offers a command prompt which executes (interprets) commands immediately.
There isn't one language that is useful for every task. Some language will always be better for specific tasks and the important thing is to recognize what these tasks are and to select a suitable programming language/environment accordingly.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Data Analysis tools

Post by caseih »

One method you could use would be to read the CSV file (one line at a time, parsing it) and then put the filed into SQLite, and then use SQL to ask questions of the data based on grouping and criteria. This can all easily be done with FB. One could probably use a CSV parsing library that was written in C from FB as well, to ease the task. And of course FB ships with sqlite bindings in the include directory.

But I agree with St_W that, depending on the task, there might be other tools more appropriate for the task.
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: Data Analysis tools

Post by paul doe »

St_W wrote:Of course this depends on the type of data analysis you want to do, but I would tend to say that FreeBasic is not the ideal language for that application area.
I beg to differ. I've coded a FreeBasic trading terminal that interfaces to the one provided by my broker via 'special' CSV files (as I simply cannot go sniffing at their port, they would cancel my account if they discover it; no buts, no questions asked) that include not only data, but commands as well (the comm is bidirectional). So, from the MT trader terminal, there's a listener that feeds data to the FB side on all the symbols available to operate (I can get by with a skipped tick or two, as my timeframe for trading is relatively long term), and the FB side sends commands to the terminal to open, close, modify positions, perform trailing stops, limit orders, whatever you can do with the terminal itself.

While I won't disclose the system for reasons that ought to be obvious, my point is that what dasyar is trying to do is perfectly possible in FB. My FB terminal has tools to perform all sorts of numerical and statistical analysis, as well as implementing some charting functionality that simply cannot be done with the original MT terminal (or any other terminal that I know of, for that matter), and a complete scripting language, based on Forth (the one from the MT trading terminal resembles C) that allows me to basically perform everything I want directly from the FB terminal command prompt. And all this is FB code, no other language was used (apart from the MT terminal listener, of course)

The real problem is: the OP has no real knowledge of how to achieve even the most basic operations (like opening a file and parsing data, not to mention the data analysis he wants to perform), for he has to code them himself. He can use a database for the heavy lifting as caseih suggested, but then he'll have the problem of how to interface with it, and off we go again with another thread like this.

@dasyar: you said that you're willing to receive any kind of comments, even skeptical ones. While mine isn't exactly skeptical, you can call it 'harsh' if you want: learn how to program first.
Start with reading the manual, and then try to do very basic things: opening a file, receiving input from a file, parsing, lexing, that kind of stuff. Then you can proceed to more advanced things like threading, data analysis (FB won't do it for you, sorry), interfacing with other apps/languages, and all that, for it seems to me like you have no idea of what you're doing at any given moment. Other members already adviced you on this (BasicCoder2 called it 'cowboy coding', which is indeed apropriate =D) but it seems that you're continually trying to bite far more than you can chew.
Last edited by paul doe on Nov 16, 2017 22:19, edited 1 time in total.
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Data Analysis tools

Post by MrSwiss »

paul doe is correct, like I've told her/him, quite some time ago, in this tread.
Especially the attitude expressed, is intresting, to say the least.
She/he obviosly wants, that others do the "coding" for her/him,
later on, claiming the code to be: my code (worst of all!).
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: Data Analysis tools

Post by dasyar »

Thank You for the robust discussion, I got my answer.
paul doe
Moderator
Posts: 1730
Joined: Jul 25, 2017 17:22
Location: Argentina

Re: Data Analysis tools

Post by paul doe »

dasyar wrote:Thank You for the robust discussion, I got my answer.
You're welcome, but no you didn't, it seems: viewtopic.php?f=2&t=26114. Exactly as I foretold.
Post Reply