How does the server scripting engine know to whom to send a new page?

[chrstrbrts]

Hello,

Let me explain my question.

Let's say a dynamic website lets clients connect to each other, even if that connection is just the ability to view someone else's profile.

You can think of this website as existing in some "site state" at any given time that is the amalgamation of each user's "client state."

Now, let's say that three users are logged in, Alice, Bob, and Carl.

Alice is viewing Bob's profile, Bob is viewing Carl's, and Carl is viewing his own.

Carl then decides to edit his profile and change his favorite food from pizza, to cheeseburgers.

He edits and then clicks "submit" which sends a signal "down the line" to the server to tell it "Hey, I changed my state in this fashion."

The site's state now needs to be changed as well.

The server scripter pulls the appropriate database and makes the required change.

Awesome.

But lets say that you want this site to let all logged in users be up to date with the current site state.

The server now has to send the info out to all logged in users.

One way is to send new versions of whatever page Alice, Bob, and Carl happen to be viewing, presumably the last page sent out to them.

But, this is inefficient.

Only Bob and Carl are viewing Carl's profile, not Alice.

Alice is viewing Bob's page, and Bob's page has not changed from the first site state to the second.

How does the sever scripter know that Alice's page should not be resent?

Is there a table kept somewhere in a database that has "users" on the vertical and "last page sent" on the horizontal (or vice versa)?

Thanks.

 

[Schmide]

Looks too much like homework.

 

[chrstrbrts]

Looks too much like homework.

I'm old as the hills and not in school. 😀

If you can help me in my quest for knowledge, please do.

 

[Gryz]

At most websites, the users will keep looking at old information forever. The page won't change on their screens. They have to click the "refresh"-button. Or they have to revisit the page later. This is very acceptable behaviour in most cases.

One solution could be to include some java-script code in the page. This java-script code will run a timer, say every 60 seconds, and every 60 seconds the browser will request the server to send the latest version of the page. Or the page could have a version-number or timestamp, and the java-script can compare those every 60 seconds. If the page has changed, the java-script can request the full page, and display it to the user. Some websites do this, but I think it's a minority.

 

[chrstrbrts]

At most websites, the users will keep looking at old information forever. The page won't change on their screens. They have to click the "refresh"-button. Or they have to revisit the page later. This is very acceptable behaviour in most cases.

One solution could be to include some java-script code in the page. This java-script code will run a timer, say every 60 seconds, and every 60 seconds the browser will request the server to send the latest version of the page. Or the page could have a version-number or timestamp, and the java-script can compare those every 60 seconds. If the page has changed, the java-script can request the full page, and display it to the user. Some websites do this, but I think it's a minority.

OK. So you're saying that the server can't send pages that weren't asked for?

Am I to assume then that the paradigm is that the client ALWAYS sends requests to the server for pages and the server sends them and NEVER takes it upon itself to force pages out to clients?

 

[Ken g6]

Am I to assume then that the paradigm is that the client ALWAYS sends requests to the server for pages and the server sends them and NEVER takes it upon itself to force pages out to clients?

That's pretty much correct. Which makes live updates like you're talking about difficult.

The client can poll for updates. The server frequently knows the last modified time for a page, and can send that out so all caches (including the one in your browser) will display cached content if possible.

The client can also smartly poll for updates. It can leave a connection open until the timeout. If it gets an update from the server before that timeout, it requests details; otherwise it closes the connection at timeout and opens a new one. HTTP/1.1 allows multiple requests on one connection, so it makes this slightly easier.

Theoretically push notifications may be possible now, but they should require a socket on the client side, so most good firewalls should block them.

 

[chrstrbrts]

That's pretty much correct. Which makes live updates like you're talking about difficult.

The client can poll for updates. The server frequently knows the last modified time for a page, and can send that out so all caches (including the one in your browser) will display cached content if possible.

OK. In this situation, what happens if the page does NOT need to be modified?

Does the server just send nothing, or does it send a "no new info" message?

The client can also smartly poll for updates. It can leave a connection open until the timeout. If it gets an update from the server before that timeout, it requests details; otherwise it closes the connection at timeout and opens a new one. HTTP/1.1 allows multiple requests on one connection, so it makes this slightly easier.

So the server CAN push something out that wasn't asked for if it's just a notification.

And keeping the connection open is done solely with JavaScript on the client's machine?

Theoretically push notifications may be possible now, but they should require a socket on the client side, so most good firewalls should block them.

Wait a minute, I think I understand a little better now.

There's a difference between requesting info and establishing a connection.

Traditionally, with HTTP the client reaches out to the server to establish a connection, not the other way around.

However, once the connection has been established, info can flow both ways without the client necessarily asking for it.

Am I right?

So, to do it my way, you would need to ditch HTTP and use a new network layer protocol.

What about UDP?

 

[Red Squirrel]

HTTP does not really have a concept of "users logged in". When a page is done loading the connection is broken, and that's that. It wont change until it is refreshed. If you want the page to auto refresh with new data you have to code that in (most likely with javascript) to poll the sever and then update the page as needed. There's lot of ways you could do it. A very crude way is a auto meta refresh but that would be super annoying for the users as if you're in the middle of reading something the page will refresh and jump all over as it loads and you lose your place.

 

[Crusty]

HTTP is uni-directional. Only the client can make requests of the server.

WebSocket is a protocol that allows for a HTTP request to be 'upgraded' for bi-directional communication, it is for instance how Gmail provides realtime updates and refreshes of their inbox.

 

[chrstrbrts]

HTTP is uni-directional. Only the client can make requests of the server.

WebSocket is a protocol that allows for a HTTP request to be 'upgraded' for bi-directional communication, it is for instance how Gmail provides realtime updates and refreshes of their inbox.

How would a server get its clients to agree to the WebSocket protocol?

 

[nickbits]

The page has to be programmed to use web sockets, it isn't automatic. I think all modern browsers support them.

 

[purbeast0]

this can easily be accomplished with polling.

 

[chrstrbrts]

The page has to be programmed to use web sockets, it isn't automatic. I think all modern browsers support them.

OK. Is this with JavaScript? PHP?

this can easily be accomplished with polling.

Polling? The client asking the server if there is anything new for the current page?

 

[Red Squirrel]

Polling? The client asking the server if there is anything new for the current page?

Exactly. This is probably the easiest and most compatible way of doing it. Just have the web page occasionally poll a server script to check for new data. There's various ways you can code this, depends on the web page. Just don't go crazy with the polling, do it like every 5 seconds or something. Heck depending on the nature of the program once a minute might be fine. Remember that if someone leaves their browser open at that page it will keep polling so if 1,000 people do it, that's a lot of requests going to your server. I'm not familiar enough with javascript code but I presume there may be ways to check if browser is active window or not, so you can check for that too. If it's not active then you just poll next time it becomes active, or something.

 

[h4rm0ny]

@OP. I think you've basically got it by now. Stay away from the concept of server pushing to the client. There are ways to do it, none are standard, all are a minefield resulting from the fact that they're working against a basic design principle of the web. Client asks, server provides.

I get the impression that you in the position of being intelligent but not having the background / knowledge. I'll therefore point you at two things that you may not be aware of but which are pertinent.

The first is AJAX. AJAX is a way of requesting information without having to update the entire page. This is what you would want for the scenario you describe. Yes, you could set a timer in Javascript that just kept re-loading the page every 60 seconds but this would be bad for two reasons. Firstly, the user may be doing something on this page (e.g. composing a private message to Bob) and resetting the state of the page every sixty seconds would be more than a little frustrating. Secondly, it is inefficient and even on a fast connection is likely to be obvious and irritating to the user. Therefore you would build the page in such a way that it made AJAX calls to the server (you do this with Javascript) to just get the information it wanted (e.g. Bob's Last Viewed Page value) and replace what is in the currently loaded page with it.

The second is HTTP codes. You will know 404 which means page not found. As you're learning web-programming, you will very soon learn 500 which means Server Error 😉. You may not know such things as 304 which means "Not modified". You can code things up such that if Bob's profile information is the same, the server returns 304. The client browser will then shrug and say "okay, I already have the latest version" and not actually download the data at all. This is more of use with larger amounts of information, but a valuable principle.

However, it is perhaps a little advanced for someone still assimilating the basics. I would focus on getting something working first, I just want to let you know that there is more out there as context is always helpful.

 

[chrstrbrts]

Stay away from the concept of server pushing to the client. There are ways to do it, none are standard, all are a minefield resulting from the fact that they're working against a basic design principle of the web. Client asks, server provides.

Really? I have to admit that the websocket approach appealed to me.

Without going into too much detail, my site will connect users together in real time. It's important that they know who is available for connection and who isn't.

My site isn't for stock trading or anything, so I don't need split second updating. But, users have to know what's happened in the last minute or so.

I need some polling / server pushing. There's really no other way.

The first is AJAX. AJAX is a way of requesting information without having to update the entire page. This is what you would want for the scenario you describe. Yes, you could set a timer in Javascript that just kept re-loading the page every 60 seconds but this would be bad for two reasons. Firstly, the user may be doing something on this page (e.g. composing a private message to Bob) and resetting the state of the page every sixty seconds would be more than a little frustrating. Secondly, it is inefficient and even on a fast connection is likely to be obvious and irritating to the user. Therefore you would build the page in such a way that it made AJAX calls to the server (you do this with Javascript) to just get the information it wanted (e.g. Bob's Last Viewed Page value) and replace what is in the currently loaded page with it.

So, what you're saying is that I can partition the page into static areas and dynamic areas?

For example, the background may be static as well as a paragraph on the left of the screen, but the area on the right needs to change according to site state. Yes?

Is this called DOM?

However, it is perhaps a little advanced for someone still assimilating the basics. I would focus on getting something working first, I just want to let you know that there is more out there as context is always helpful.

I agree. I'm starting deep and moving towards superficial.

That is, I'm starting with SQL / MySQL then on to PHP, JS, CSS3, and finally HTML5.

My first goal is to build a database.

 

[Ken g6]

My site isn't for stock trading or anything, so I don't need split second updating. But, users have to know what's happened in the last minute or so.

If you only need to know once a minute, I'd definitely avoid the complications of push updates. Like my Slashdot sig says (in code), "Premature optimization is the root of all evil" (Donald Knuth).

So, what you're saying is that I can partition the page into static areas and dynamic areas?

For example, the background may be static as well as a paragraph on the left of the screen, but the area on the right needs to change according to site state. Yes?

Is this called DOM?

The DOM is a tree structure made from the nested HTML tags on a page. One idea would be to just update a chunk of that structure. You might call this an AHAH moment. 😉

That is, I'm starting with SQL / MySQL then on to PHP, JS, CSS3, and finally HTML5.

Personally, I'd want to see something working on my screen first. But to each their own.

 

[h4rm0ny]

Really? I have to admit that the websocket approach appealed to me.

Okay, then let me rephrase things in less uncertain terms: "You do not want to do this."

There are things you know. There are things you don't know. Then there are things you will regret knowing. Just trust me, get the hang of the normal way of doing things first. A piece of advice that will serve you well: use the simplest approach that meets your needs. Don't try to build a jet when a car will do.

Without going into too much detail, my site will connect users together in real time. It's important that they know who is available for connection and who isn't.

My site isn't for stock trading or anything, so I don't need split second updating. But, users have to know what's happened in the last minute or so.

Sixty seconds is not a problem. Have the page ping your server every forty seconds or so and then keep a record on your server (e.g. in your database) of the last time you heard from a user. If it's more than this time, then treat them as "Away". You can use their request for a list of online users as your "ping" if you like, given that they'll need to keep asking this to stay up to date themselves.

I need some polling / server pushing. There's really no other way.

Web servers do not push. It's how the web is designed. There are exceptions and ways around that, but you're fighting against the design goals of the Web. The standard approach is to have the client keep pinging the server with requests. That way the server knows the client is still out there connecting. Well at least with a certainty of sixty seconds or however often it is between pings.

So, what you're saying is that I can partition the page into static areas and dynamic areas?

For example, the background may be static as well as a paragraph on the left of the screen, but the area on the right needs to change according to site state. Yes?

Yes, that's the principle. You might have a DIV whose contents keeps updating with the latest message for example.

Is this called DOM?

Not quite, but it's related. The DOM (Document Object Model) is the structure of a web page (practically speaking). So a DIV is an element within the DOM. So is a <P/> element or an <span/> element. So when you do updates in this way, you are updating the DOM by updating elements within it.

I agree. I'm starting deep and moving towards superficial.

That is, I'm starting with SQL / MySQL then on to PHP, JS, CSS3, and finally HTML5.

This is good design. Work out the structure of your data first by building your database and the queries you will need. Then work out the exposing of that data - put it on a web-page using PHP. The finally work out the presentation of your data (styling, javascript, et al.). You have a good working order here.

My first goal is to build a database.

A laudable goal. There are few things in life that cannot be solved with this.