Finance Data API

[rustyuk]

Currently developing a Robo-Advisor but I'm running into difficulties in respect of data access.

I'm currently using a mixture of MorningStar (via Rapid-API) and Yahoo Finance - But neither seem to cover UK EFT's or Funds such as Vanguard 80.

IG.com seems to other the most complete service but I wondered if I am missing an obvious data source (Paid Subscription is fine)

[anonymous-user]

Anything useful here?

https://www.londonstockexchange.com/products-and-s...

[DonkeyApple]

rustyuk said:

Currently developing a Robo-Advisor but I'm running into difficulties in respect of data access.

I'm currently using a mixture of MorningStar (via Rapid-API) and Yahoo Finance - But neither seem to cover UK EFT's or Funds such as Vanguard 80.

IG.com seems to other the most complete service but I wondered if I am missing an obvious data source (Paid Subscription is fine)

Data feeds are a nightmare. Most of the free APIs use data from the dark pool exchanges such as Chi-X so the feed splits are different and not everything isnon there. They also have non reselling agreements or restrictions and Murphey’s Law also dictates that one of the data sets you’ll need is one of the comically expensive ones or you just need it for a couple of markets.

The other problem you will have is that on the investment side some of the stuff you will want pricing on is very low volume and the dark pool may have no transactions in it for over an hour so the tick data halts and on some feeds needs a manual kickstart to get pricing again.

[rustyuk]

Thanks folks, I came to the conclusion that I had to create a couple of data ingest feeds to cover what I needed.

It's just a self learning project more than anything to test out some of Mark Shipman's strategies of very simple moving averages against my current investments and the main market indices. I can of course just use the standard charts available on a couple of platforms but I wanted to develop this as an automated solution.

Been meaning to do this for over a year, it's been an expensive mistake as the sell signal was confirmed around the end of Feb for all of my pension investments. Nasdaq 100 is the only index that has returned to buy / hold using Mark's technique.

Using my tool I have tested the strategy against a large number of funds and indices historically and it's generally correct. A little late sometimes in generating the sell signal. I've added a couple of other very simple features and it seems to be performing well.

Edited by rustyuk on Thursday 30th April 14:26

[NickCQ]

If you really must day trade your pension, do it properly and rent a Bloomberg terminal. There’s a reason that the industry uses it.

[nyt]

Google finance is a good place to start.
It has prices for all of my ETFs - but I can't be certain that it has them all.

I use a google spreadsheet to get the prices sheets.google.com

I then is google finance formulae to populate with prices. Eg:

=GOOGLEFINANCE("LON:LLOY","PRICE")

You can also scrape prices from other web pages into cells where the data isn't available

= index(ImportHTML("https://uk.investing.com/funds/fundsmith-equity-t-acc-historical-data","table",1),2,2)

It's google so there's always a risk that they will withdraw the service

[rustyuk]

Sorry not day trading, it's just using moving averages to react to market conditions. Close of Week SMA evaluation so only a few trades a year.

Have been using both Google and Yahoo - lots of support for Google is being removed or deprecated from the Python libraries. I rolled my own HTTP request for Google but was unable to get historic fund data.

I was naively hoping for a single API provider. IG seem to have the best overall coverage but their API uses OAuth rather than a simple token in the header so a little more involved when writing code from scratch which, was the whole purpose of the exercise really.

The aim is to have a cloud native app that monitors my small SIPP, sending notifications when it thinks I should review my positions. Might even create a nice UI if I get more bored than I am now

Really recommend Mark Shipman's book Big Money Little Effort - Quite a few people are angry when the strategy is revealed and it can be explained in a single sentence.

What stands out for me is Mark's admission then he has spent lots of money on the best research possible, hiring very talented mathematicians. Not a single strategy from this research could beat SMA over a long period.

Again I'm not using this for trading I only have a small SIPP (less than 100k) and it's invested in the usual suspects such as Vanguard 80, iShares Blackrock etc. However, if I had used the signals it would be substantially more than 100k now - oh well ....

[rustyuk]

If anyone can get the Vanguard 80/20 Life Strategy historic data from Yahoo or Google then I would be very grateful if they could share the request details.

I've also tried MorningStar but it seems the API only supports data that is available via the .com Website. Tried a number of combinations and get back a 404 or an empty vector.

[i4got]

rustyuk said:

If anyone can get the Vanguard 80/20 Life Strategy historic data from Yahoo or Google then I would be very grateful if they could share the request details.

I've also tried MorningStar but it seems the API only supports data that is available via the .com Website. Tried a number of combinations and get back a 404 or an empty vector.

I don't have access to any morningstar premium data that might include historic prices.

However you can try something like

=IMPORTHTML("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000MLUQ", "table", "4")

or for historic returns

=IMPORTHTML("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000MLUQ&tab=1", "table", "7")

The latter gives this -

[rustyuk]

Many thanks - Is that screen scrapping the html?

[i4got]

rustyuk said:

Many thanks - Is that screen scrapping the html?

It's just a function called within a GoogleSheets cell.

You can wrap Index function around it to get specific data elements from the table.

eg

=Index(IMPORTHTML("https://www.morningstar.co.uk/uk/funds/snapshot/snapshot.aspx?id=F00000MLUQ&tab=1", "table", "7"),3,2)

Will return -

1.86

- which is the 3rd row second column

[rustyuk]

Understood! Cheers!!