Programming Question - Realtime HTTP Stream - Thoughts?

[TheExcession]

I'm wondering about creating a plug-in to Apache such that I could point my browser at a URL, the URL would then point to a file on disk, but the file on disk is being written to in real time.

I guess the nearest equivalent would be like Syslog, but I'd like to use HTTP as a transport.

So instead of tailing a syslog file, the information would be streamed out to a browser, slowly filling up the browser window, line by line as the information is generated.

Platform is Linux RedHat

Has anyone come across anything like this?

cheers
Ex

Edited because I forgot to say target platform is RedHat Linux

>> Edited by TheExcession on Thursday 15th December 17:26

[BliarOut]

First question has to be what's the file writing? There has to be a ready made way to do what you want.

[GreenV8S]

Not exactly, but depending what platform you're on could you have your server map the URL to a named pipe, or summat like that? You'd need to do a bit of work client side to render the raw text stream for you but that isn't exactly rocket science. As a fallback I guess you could try and do it all yourself using the sort of services that streaming video servers use, but that seems like rather hard work.

[TheExcession]

BliarOut said:

First question has to be what's the file writing? There has to be a ready made way to do what you want.

The file contains text lines, each line is generated by a modded copy of tethereal which is monitoring control messages in a satellite ground station.

The controll messages are instructing the satellite to allocate different frequencies to different spotbeams that are shining back down to earth.

So far I have this written....
(sorry bout the size of the image)



You can see the spotbeam map, all unlit at the moment, though they do change colour as event messages come through.

What we want to do is supply the state change messages that this my App uses to third parties.

Due to the usual Firewall and Security crap, I'd like to use HTTP as the transport...

So I want a broswer to be able to connect to a 'tail' of the state file and have the stream of messages go back to the browser.

There are a few gotchas, the state files get big pretty quick, especially if there is a lot of activity on the satellite, so we roll over to a new state file every 15 minutes.

There is a marker at the end of the current file to tell you what the next file is, so it will need to be able connect to the current file, then hop over to new files as and when they are created.

Now, like I said it's a bit like syslog, and infact it will likely be pretty simple for me to put a syslog client into tethereal, but that sill leaves me needing to connect Apache to to the 'syslog' file so I can use HTTP as a transport.

Failing that I'll just have to write a thin HTTP Server, or as I said put a plugin into Apache

If you get my drift....

cheers
Ex

[TheExcession]

Oh, sorry forgot to say, if anyone mentions xml I'm gonna come round and burn the bastard chair out from under your lazy arse.

(just wanted to get that straight )

[Don]

How I'd do that is this:

Write a Web Service that you can consume with the built-in XML parser in Internet Explorer - you can access it from JSCript. Have a JScript function on your hosting web page simple poll the web service from time to time and have it send you the data.

Or you could use an ISAPI DLL you access as a URL - since its written in C++ and running as a service you can have do whatever the hell you want. Have the aforementioned XML parser poll that using a JScript loop.

Help at all as ideas?

We do something like this all the time to dynamically access SQL data from the client side in JScript without page turnaround. We use SQL Server's built in XML Web Service on the server and some custom JScript in the pages. Works a treat.

Oh..and ALL of it is over the HTTP protocol - no probs with firewalls.

>> Edited by Don on Thursday 15th December 17:12

[TheExcession]

Don said:

How I'd do that is this: Write a Web Service that you can consume with the built-in XML parser in Internet Explorer - you can access it from JSCript. Have a JScript function on your hosting web page simple poll the web service from time to time and have it send you the data. Or you could use an ISAPI DLL you access as a URL - since its written in C++ and running as a service you can have do whatever the hell you want. Have the aforementioned XML parser poll that using a JScript loop. Help at all as ideas? We do something like this all the time to dynamically access SQL data from the client side in JScript without page turnaround. We use SQL Server's built in XML Web Service on the server and some custom JScript in the pages. Works a treat. Oh..and ALL of it is over the HTTP protocol - no probs with firewalls. >> Edited by Don on Thursday 15th December 17:12

Hi Don,

First mistake you said xml......

However, in my post 'browser' should really read any HTTP client, so at the end of the day, it's never going to be a 'browser', nor is it ever going to be anything that is 'xml intelligent', it's just something that can read from a socket pointed to port 80.

I just changed my post, as you mentioned ISAPI - target platform will be Linux, Target Web Server will be Apache unless I write my onw thin client (more I think about this the more I think it it is the simplest solution)

It's my understanding that most browsers will just render what ever they receive out of this port, and it will be our third party's responsibiility to decode what ever we send them, again, there is no where I'm going to mark up our event messages into xml or anything else.

I just wanna be able to say to them, Get this URL on our web server and we'll start sending you the stream.

A bit like telneting to a web server on port 80 and issueing a GET / it returns the default web page, only now I want to send GET /spotmap_states - the server will connect to relevent file and start streaming data written to that file in realtime, keeping the connection open.

I like the named pipe idea further up the thread, anyone know how to conenct a URL to a named pipe in Apache? Then I can test it with /var/log/messages

cheers
Ex