Help Please! IB nightmare.

IB recently FORCED me to upgrade to a newer version of TWS.

I don't mind that the old software is no longer supported. I want to use it and have asked several times to be allowed to - no dice. The old software it updates the charts and doesn't crash.

My day is now a nightmare.


I can't convince them to let me use the old software and they want me to debug the garbage they are shoveling now. Laughable. (well not really laughable ..pull-out-my-hair-able!!)

The only reason I have IB is that the data feed for futures is included if my commissions are above a certain level =$30/mo. I trade ES (the SP mini) only. If I lived in the USA I would just close my IB account and get Trade Station - but according to their rep the Trade Station brokerage service is not available to non-US citizens.

I've tried using Ninja with IB hoping to get charting and then place the trade separately.

I want to back-test at Ninja. Working with the Ninja reps they don't seem to be able to understand that the RTH daily data is stored natively at IB and offer the clumsy workaround of 405 minute bars instead of daily RTH.

As this takes a lot of data (the minute bars) backtesting at Ninja on strategies that require RTH OHLC doesn't work with this 405 minute workaround. (IB times out Ninja with too much historical data request and Ninja doesn't keep what it had and start again from the IB lock-out.)

So, I need some help from fellow traders:

1. What suggestions do you have for my IB - garbage software problem? Do you know of a brokerage that provides the futures data at low cost I could use instead?

2. Does anyone who uses Ninja with IB data feed know how to set up the "instrument" (ninja speak for security) parameters for daily RTH ES data. That is I want to add in an instrument in Ninja that is "ES RTH Z09".

Thanks in advance,

I use open futures Ninja which is $50 a month through AMP. I also recommend gotrade.us transact which is free for DOM charting is $20 more a month. There charts are okay but ninja is better.

I didn't have enough time earlier....but you can change ninjas 24/5 to RTH by right clicking the chart/properties change the session begins and session ends. btw Esignal is a great charting software/direct access trading through most brokers and its reasonably I think there like $49 bucks for emini's only. I loved them but for some reason only them were slow on my computer...I would run as many as 30 charts and it would freeze all the time. But it works fine with like 5 or less charts. I needed someone who does not freeze up during heavy voulme. esignal blamed it on my internet (T1 line couldn't handle it I guess) and computer (super computer just for trading) So I droped esignal and picked up meta trader...there all right. And ninja, there very good. And transact, of which I have never had a data clog even in the most bottle necked days. I now spend less than $100 a month for data, I used to spend over $600 a month. I figured as much money I have spent on data I could have had a brand new Hummer with 2 years gas! Thats a lot for data.

Hi CJ

I'll investigate your leads, thank-you.

Re Ninja RTH: the start and end of session only works for 1 minute charts which is the limitation I was speaking of above.

To clarify the workaround is OK for "daily" charts only going back a short period of time. However, backtesting a strategy that requires RTH OHLC data for (daily CCI and ATR gap fade etc.) is where the problem arises.

This is because the data vendor (IB) only keeps 1 minute data for a limited period of history. A second problem is that ninja has to download a huge amount of data to remake daily HLOC values and IB locks out the Ninja request after a certain amount of 1 min data retrival(at least for me it does!).

I think you may be using the wrong tools for the job your attempting to perform.

Either Tradestation or E-Signal are designed to do extensive backtesting of intra-day data.

In Ninja, as Joe suggested, the start and end time of the session works on any time frame chart I use it on. For example,I use RTH start and end times on my 60 minute candle chart of the ES and it works fine, but I am using the Zen-Fire data feed. The problem you are having may be IB data specific with respect to Ninja.

Another option you might investigate is something like Ensign software which plays well with IB data, has a large base of IB users, and excellent support. Also a lot of folks use Sierra charts with IB.

Hi PT,

Thanks.

No doubt you are correct in that I am using the wrong tools! my hair loss confirms your suspicion.. As I mentioned in my initial post Trade Station is not available to Non-USA residents.

Re ninja OHLC : Just to be sure we are taking the same thing, are you saying if you tried a back-test daily CCI cross-down over 3 years you would not have a data problem with Ninja?

I am confident that:
1. IB has daily data going back for several years
2. If I knew the Ninja instrument parameters to call the daily RTH series I could specify that data series from IB.

I have confirmed with Ninja that start and end time only works for minute data and does not apply to daily data.
---email 1---
Hi Chris,

I trade the ES mini and I need the RTH data.

It seems data providers (such as IB) provide the daily RTH (regular trading hours) as a different data series than the ETH (extended trading hours) daily. How do I specify the RTH daily data series?

Many studies required RTH daily such as ATR, CCI, and many trading strategies also required daily RTH yet it does not seem available (specifying session times on daily chart). Why not?

The work-around you offerred of 1440 minutes does not work. This is because most data feeds only keep minute data for a short historical time period. They keep 15min data for longer times, 30 minutes longer again, etc 60 min and daily.

To have 3 years of RTH daily data with a daily datafeed is about 660 data bars (220 times 3 yrs). To keep it for 1440 minutes is 950,400 data bars. If Ninja even offered 15min data bar download that would decrease it to 63,360 data bars.

-------email 2----
Here is what happens with IB on your 1440 minute idea.

{picture of historical data retrieval request abort by IB part way through data retrival}

--
I agree it works for a short period of time

Hi PT

Do you use Ensign currently?
(From a site where users post their reviews)

"Dumping Ensign after years" - opm8 November 12, 2009 4:35 PM

"Ever since the redesign of Ensign over a year ago it has gone steadfastly downhill. The almost daily updates include bug fixes introduced by the previous version. It's a never-ending circle of "improvements" coded by someone who it seems understands nothing of coding best practices and UI design. In-program and on-line documentation is vastly behind the latest features so they've taken to posting answers to user questions as their "documentation." If you ever ask a support question the answer always includes installing the latest version which fixes some previous bug. I will say it has some good drawing tools, which is its only positive feature. However, I'm sick of the frequent crashes and frustrating, extended 100% cpu spikes that freeze the whole computer for minutes at a time. When it goes down it takes about 5 restarts before it comes up without crashing after doing a massive, needless automatic refresh of everything under the sun (even though all charts were up to date before the crash). I've optimized the living crap out of it, use only minimal look back periods in my charts and a couple of ultra-simple indicators. And at $50/month they can shove it. It used to run on a dual-core 2.67Ghz cpu and 4GB of RAM. Buh-bye pos buggy platform. "

Hi Blue -

How Ninja works is going to depend on the data provider. For example, the ZenFire data feed does not provide historical daily data. ZenFire provides very limited historical intraday data (bare minimum). TransAct feed, same problems. For those reasons it would be very difficult if not impossible to do the type of extensive backtesting using Ninja with Zen Fire (or TransAct) data.

How Ninja works with the IB historical database is outside the scope of my experience unfortunately. Also, I thought IB provides symbols for RTH data ?

If you feed eSignal into Ninja it will have access to years of daily data, and also access to the eSignal historical intra-day database. eSignal provides RTH symbols. Of course, if you are paying for eSignal data it may make sense to just use the eSignal strategy testing facility and bypass Ninja entirely. You might consider using eSignal for your strategy testing phase, then once your system/method/strategy is worked out to your satisfaction, drop it and find something faster and cheaper to support your actual trading.

With respect to Ensign: the software has frequent updates, primarily to add new features and fix bugs. In terms of the stability issue mentioned, I have seen the occasional chart/data crash and the program has to be restarted and charts/data reloaded. In terms of system or strategy development and backtesting, Ensign may not be the ideal solution however. Ensign is good for things like real-time analysis of Fibonacci levels and complex chart patterns (harmonics). Also their MP charts are good.

For someone focused on discretionary trading however (once you get free of the development and testing phase), Ninja (DOM) + Ensign (Charting) makes an excellent combination.

You may want to look into something like Strategy Runner which is designed for system development and extensive system testing. There are other programs like Strategy Runner out there, and this may be the direction you may want to research. Keep in mind that most retail charting packages which provide backtesting, do so as an after thought and are not that good at doing it. For example, Ninja started out as an alternative to the very expensive TT DOM ladder (which cost $600 a month to lease at the time Ninja arrived on the scene). Ninja recently added charting as an add-on feature, and now has added a system testing capability. These (charts and systems) are afterthoughts for Ninja and thus a serious tester will quickly run into the inherent limitations of the now burgeoning Ninja (we can do it all now) program.

Hi PT

Thank-you very much for your detailed reply which is very helpful.

Taking you wise advise, and combining with my situation, I will:
1. test as best I can on Ninja and within my spreadsheet of RTH daily data,
2. use IB for the data feed ($10/month) and Ninja for the charts (free)
3. maintain my spreadsheet tracking
4. backtest as best I can on ninja until I have the confidence needed to trade.

As I am very low on capital, I feel I must be extremely careful with my funds remaining so that I can last until I produce breakeven income from my trading.

Hopefully the flutter of butterfly wings of Dubai World signal the long awaited downturn!
http://news.goldseek.com/PeterCooper/1259334875.php
"Shocks in emerging markets like Dubai are the flutter of butterfly wings that produce a hurricane elsewhere, and $59 billion is a bit more than a butterfly. Investors should exit all stock markets and buy bonds or precious metals or short emerging markets. Gold hit $1,195 as this article was written."

------blue-----

Originally posted by pt_emini
How Ninja works with the IB historical database is outside the scope of my experience unfortunately. Also, I thought IB provides symbols for RTH data ?

==>I believe IB has (hidden somewhere) a symbol for RTH data and finding this and entering it into Ninja is what I am looking for. I have emailed Ninja on this question and with much explaining and emailing, to a muttonhead, my request has been passed, supposedly, to their programmers.

Originally posted by pt_emini
Of course, if you are paying for eSignal data it may make sense to just use the eSignal strategy testing facility and bypass Ninja entirely. You might consider using eSignal for your strategy testing phase, then once your system/method/strategy is worked out to your satisfaction, drop it and find something faster and cheaper to support your actual trading.

===> good idea.

Originally posted by pt_emini
For someone focused on discretionary trading however (once you get free of the development and testing phase), Ninja (DOM) + Ensign (Charting) makes an excellent combination.

=======> thanks it is good to your thought on the final configuration goal!

thanks again