How to make XmlHttpRequest calls to another server in your domain
This post is more technical and detailed than what I usually
write on fettig.net. I’m going to talk about a technique I’ve been
working on to work around a limitation in the XmlHttpRequest object
used in Ajax applications. Web geeks, please read on.
Updates to this post
See this post for an updated version of this techinique that works (hack-free) in Firefox 1.5.
The problem
XmlHttpRequest objects are bound by the
same origin security policy
of browsers, which prevents a page from accessing data
from another server. This has put a serious limitation on Ajax
developers: you can use XmlHttpRequests to make background
calls to a server, but it has to be the same server that served up the
current page. Known workarounds for this limitation involve either
server-side reverse proxying
or
bypassing XmlHttpRequest entirely.
In my case neither of these approaches was
going to work. I wanted to use
LivePage, the live-update framework
developed by Donovan Preston and other brilliant hackers at
Divmod. LivePage
works because it uses Twisted,
which is good at handling lots of long-lasting network connections at
the same time. Since Apache isn’t good at handling lots of long-lasting
connections, putting an Apache reverse proxy in front of the Twisted
server would put a major hurt on performance and scalability. And
since LivePage is bound to XmlHttpRequest, I couldn’t use a
non-XmlHttpRequest alternative.
Our approach with JotSpot Live has been to let a Twisted server
handle all page requests and XmlHttpRequest calls. That’s fine as
long as JotSpot Live is a standalone, dedicated web site. But what our
customers are asking for (and what we want) is the ability to
have Live-style realtime updates in ordinary xxx.jot.com sites. And we
don’t want to put a Twisted server in front of every site in our
domain. So I’ve been trying to find a way to let any page in our domain
communicate with live.jot.com through XmlHttpRequests. As it turns out, it is
possible, but you have to jump through some hoops.
A couple of notes on the examples here:
- I’m not really using two different servers. In my examples, a
page on http://fettig.net/ is attempting to make an XmlHttpRequest call
to http://www.fettig.net/. The fact that my Apache setup maps these to
the same virtual host makes no difference to a browser, though: they’re
not the same hostname, and they’re treated no differently than if they
were two different sites with different content. The results would be
the same if I was trying to make XmlHttpRequest calls to an
ajax.fettig.net server with its own IP address. - I’m using a simple XmlHttpRequest wrapper library that provides a getUrl
function. getUrl takes a URL and a callback function, opens an
XmlHttpRequest connection to that URL, and calls the function with the
results. If you’re interested you can view the full code here. - The service to which I’m making XmlHttpRequest calls is a simple
PHP page, ajaxdata.php, that prints out the current UNIX time on the
server: Not particularly useful in real life, but good enough for testing purposes.
First attempt: the naive approach
Here’s the first thing I tried: a standard
XmlHttpRequest with the full URL of a page in a different subdomain. I
was pretty sure this wouldn’t work, but I wanted to verify it for
myself:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function getTime(){ getUrl(AJAX_URL, gotTime); } function gotTime(status, headers, result) { document.getElementById(’time’).innerHTML = result; setTimeout(getTime, 1000) } window.onload = getTime; </script> </head> <body> <div id=”time”></div> </body></html>
This page is served from fettig.net, and tries to make an
XmlHttpRequest call to www.fettig.net. In real life, these are the same
server, but the browser doesn’t know that. Unsurprisingly, this doesn’t
work in any browser. You get a security error such as this one (from
Firefox):
Error: uncaught exception: Permission denied to call method XMLHttpRequest.open
You can try this for yourself here.
Second attempt: using an iframe and document.domain
On my last trip to the JotSpot office Alex pointed me toward iframes and the document.domain
property. iframes are similar to XmlHttpRequests in that they load data
into the current page from another page. But iframes aren’t limited to
pulling pages from same web server as their containing page - they can load any URL. To prevent cross-site security
problems, browsers enforce the same origin policy in the javascript
object model: scripts running in one frame can’t access any objects
inside another iframe, unless both pages came from the same server.
There’s an exception to this rule, however. If both pages come from the same parent domain, and
both of them set the property document.domain to the same parent domain,
scripts running in either frame will be allowed to talk to each other.
For example, say the page http://www.example.com/ loads the page
http://ajax.example.com/ in an iframe. Since both pages are in the
domain example.com, if both set document.domain to “example.com” they
will be be given the ability to programatically access each other’s
data. While there are a few differences
So, can you use an iframe with document.domain
to make XmlHttpRequest connections? Yes, with two restrictions:
- The iframe must be served from the server to which you’ll be making XmlHttpRequest calls.
- You have to open the XmlHttpRequest connection before you set document.domain.
Here’s the code I used. First, the page test2.html:
<html> <head> <script type=”text/javascript”> document.domain=”fettig.net”; function gotTime(result) { document.getElementById(’time’).innerHTML = “Server timestamp: ” + result; } </script> </head> <body> Single XmlHttpRequest. Works in all modern browsers. <div id=”time”></div> <iframe src=”http://www.fettig.net/playground/ajax-subdomain/test2-iframe.html”> </iframe> </body></html>
This script in this page sets document.domain, and defines the function gotTime
to handle the results of the XmlHttpRequest call. All the
XmlHttpRequest stuff happens in an iframe, though. Here’s the code in
test2-iframe.html:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function gotResult(status, headers, result) { document.domain = “fettig.net”; // set d.d before talking to parent frame window.parent.gotTime(result); } getUrl(AJAX_URL, gotResult); </script> </head></html>
test2-iframe.html makes the XmlHttpRequest call to
www.fettig.net. It’s allowed to do this because it was served from
www.fettig.net, so it’s just talking to its originating server. Once it
gets the response back, it sets document.domain to fettig.net,
to match the document.domain value of the parent frame. Now it’s
allowed to talk to the parent frame using javascript, so it’s able to
call window.parent.gotTime.
The key is to do this in the right order. Once you set
document.domain (to anything other then the actual host the page was
served from) you lose the ability to make XmlHttpRequest calls. So
you’ve got to take care of your XmlHttpRequest business before you set document.domain and to enable communicating with the parent frame.
Follow those rules, though, and you’re good to
go. This technique seems to work in all modern browsers (I tested in IE
6, Firefox
1.0.7, Safari 1.3, Opera 8.5, and Konqueror 3.4), and it doesn’t contain anything I’d consider a hack. You can try it yourself here. So far so good!
Third attempt: repeated XmlHttpRequests
I was happy to have figured out a way to make
a single
XmlHttpRequest to another server in our domain. But as presented so
far, this technique has a pretty severe limitation: You can only make
XmlHttpRequest calls up to the point where you set document.domain.
Once you do that, you gain the ability to communicate with the parent
frame, but you lose the ability to make future XmlHttpRequest calls.
This wasn’t good enough for LivePage, which needs the ability to
continuously make XmlHttpRequest calls to the server and handle the
results. I needed a way to let the iframe communicate with both the
server (using XmlHttpRequests) and the parent frame (which it can only
do by setting document.domain).
This raised the question: is setting the document.domain
property one-time-only, or is it possible to switch it on the fly? If
it can be changed back and forth, it might be possible to change it
back and forth as needed. Here’s test3-iframe.html, a modified version of the iframe page:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function gotResult(status, headers, result) { var oldDomain = document.domain; document.domain = “fettig.net”; window.parent.gotTime(result); document.domain = oldDomain; setTimeout(getTime, 1000); } function getTime(){ getUrl(AJAX_URL, gotResult); } getTime(); </script> </head></html>
Like in test2-iframe.html, the gotResult
function in
test3-iframe.html sets document.domain to fettig.net (to enable access
to the parent frame) and then calls window.parent.gotTime. But first it
stores the current value of document.domain, which defaults to the host
the page was served from. After it’s finished working with the parent
frame, it sets document.domain back to the original value, which
restores the ability to make XmlHttpRequests. It then uses setTimeout
to set up another XmlHttpRequest call. The result should be a continually updated timestamp on the main page.
You can see this in action at
http://fettig.net/playground/ajax-subdomain/test3.html
.
This page makes repeated XmlHttpRequests, with the iframe calling a
function in the parent page each time it gets a result from the server.
Unfortunately, while this technique works great in IE, Safari, and Konqueror, it doesn’t work in Mozilla and Opera. The
first XmlHttpRequest will work, but when you try to change
document.domain back to the original value you get an error. From Firefox:
Error: [Exception... "Illegal document.domain value" code: “1009″ nsresult: “0×805303f1 (NS_ERROR_DOM_BAD_DOCUMENT_DOMAIN)” location: “http://www.fettig.net/playground/ajax-subdomain/test3-iframe.html Line: 13″]
Mozilla and Opera are more strict about the value of
document.domain - it can only be set to the current value or a
higher-level domain. For example, if the host is aaa.bbb.example.com, I
could set document.domain to bbb.example.com. At that point I could
change it again, to example.com, but I couldn’t change it back to
aaa.bbb.example.com. Once you move to a higher-level domain, you’re
stuck there.
Fourth attempt: Making it work in Mozilla
I could live without Opera, but not Mozilla. So I set out to find a way - any
way - to work around this problem. After thinking about it for a while
I started wondering: just how strict are security limitations on
communications between frames? We know that browsers don’t let frames
access each other’s objects unless they have the same document.domain.
So setting a child frame’s document.domain to something different from
the parent’s and then trying to look at window.parent.foo isn’t going
to work. But what if you could set an attribute in the child frame that
pointed to a function in the parent frame, and then changed the document domain? Would the child frame still be able to call that function?
The answer is yes, and that was the break I needed to get this thing working in Mozilla. The trick is to use two frames:
parent window bridge iframe child iframe
The bridge iframe and child iframe are both served from the server
to which you’ll be making XmlHttpRequest calls. The parent page code is
the same as in the previous examples: it defines a function to handle
the XmlHttpRequest result, and loads an iframe, this time containing
the bridge code, which looks like this:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> function gotTime(result) { window.parent.gotTime(result); // pass result up to the parent } window.onload = function(){ var subframe = document.createElement(’iframe’); document.body.appendChild(subframe); subframe.src = “test4-iframe.html”; subframe.contentWindow.bridgeGotTime = gotTime; document.domain = “fettig.net”; } </script> </head> <body></body></html>
When this frame loads, it loads another frame inside itself.
Since the child frame is loading from the same server as the bridge
frame, the bridge frame can access the child frame’s objects through
script. The bridge frame takes advantage of this to set subframe.contentWindow.bridgeGotTime to its own gotTime
function. Next, the bridge frame changes its document.domain to match
that of the parent window. At this point the bridge frame loses the
ability to talk directly to the child frame, since their
document.domain properties no longer match. But the child frame retains
the ability to communicate with the bridge frame through the
bridgeGotTime function, which was set up first. And since the bridge
frame’s document.domain now matches the parent frame’s document.domain,
the bridge and parent can communicate freely.
In my limited testing, this techinique only seems to work on
Mozilla-based browsers (I haven’t found a solution for Opera yet, but I
haven’t spent much time on it). My final example uses a hybrid approach
that tries to use document.domain switching, and then falls back on the
bridge-iframe approach if the browser raises an error when you try to
reset document.domain. Here’s the main page code:
<html> <head> <script type=”text/javascript”> document.domain=”fettig.net”; function gotTime(result) { document.getElementById(’time’).innerHTML = “Server timestamp: ” + result; } </script> </head> <body> <div id=”time”></div> <iframe src=”http://www.fettig.net/playground/ajax-subdomain/test4-iframe.html”></iframe> </body></html>
Same as everything else so far except that the iframe is loaded from test4-iframe.html. Here’s the code in that page:
<html> <head> <script type=”text/javascript” src=”xmlhttp.js”></script> <script type=”text/javascript”> var AJAX_URL=”http://www.fettig.net/playground/ajax-subdomain/ajaxdata.php”; function getTime(){ getUrl(AJAX_URL, gotTime); } function gotTime(status, headers, result) { var oldDomain = document.domain; if (window.bridgeGotTime) { window.bridgeGotTime(result); } else { document.domain = “fettig.net”; window.parent.gotTime(result); } try { document.domain = oldDomain; setTimeout(getTime, 1000); } catch(e) { // denied access to switching the domain, use bridge instead document.location.replace(”test4-bridge.html”); } } getTime(); </script> </head></html>
This code tries to switch document.domain back and forth as needed to
communicate with both the XmlHttpRequest server and parent frame. If
this fails, however, it loads the bridge iframe in its place. To keep
from having two different pages doing the same thing, this page also acts as the child of the bridge frame. If it sees a window.bridgeGotTime attribute it knows it’s running under the bridge iframe, so it calls window.bridgeGotTime instead of trying to talk directly to window.parent.
You can see this in action in test4.html. It works in all the latest browsers except Opera. Update:use test5.html instead. See this post for details.
End result: it’s possible to make XmlHttpRequest calls to another
server in your domain. You can make repeated calls too, if you’re
willing to deal with a little setup overhead. This has a number of
possible benefits:
- You could have a dedicated Twisted server handling LivePage
requests, while the main content of pages if served by Apache or some
other web server. - You could have a dedicated server for handling web services, including XmlHttpRequests
- You can use DNS to free XmlHttpRequest from the
two-max-connections limit of browsers. Browsers don’t let you have more
then two open connections to a server at a time - this gums things up
if you’re using an app like JotSpot Live and try to have more then one
tab going at a time. But if the XmlHttpRequests don’t have to be going
to the same server, you can use wildcard DNS and set each page to make
XmlHttpRequest calls something like randomhash.livepage.yourdomain.com.
If you’re read this far, thanks for hanging in there! I
hope this is a useful tool in your bag of Ajax tricks. I’m sure there’s
more to be explored here, but having gotten this
far I wanted to write this post and put it up for comments. If you
think I’ve overlooked something, that there’s a better way, or that all
of this is an ugly hack, please tell me about it in the comments.
[…] my co-worker and developer of the uber-useful JotLive, had just published a description of how to get XMLHTTP working cross-domain […]
Pingback by planet::dojo » Blog Archive » Cross-Domain XMLHTTP — November 28, 2005 @ 1:55 pm
Abe, this is good stuff. Thanks for figuring this out.
Comment by Brad Neuberg — November 28, 2005 @ 2:25 pm
How to make XmlHttpRequest calls to another server in your domain
This Abe Fettig’s Weblog � How to make XmlHttpRequest calls to another server in your domain could open some great new hosting options allowing static sites to add dynamic features from 3rd parties (by point subdomains to the 3rd parties).
Trackback by 42 — November 28, 2005 @ 9:03 pm
Abe Fettig?s Weblog � How to make XmlHttpRequest calls to another server in your domain
This post is more technical and detailed than what I usually
write on fettig.net. I’m going to talk about a technique I’ve been
working on to work around a limitation in the XmlHttpRequest object
used in Ajax applications. Web geeks, please read o…
Trackback by ADC Memory Bank — November 29, 2005 @ 2:17 am
Have you tried mod_pubsub?
Comment by Dan — November 29, 2005 @ 3:10 am
This seems like a great idea for use on sites to which you don’t have server access (i.e., my one hosted website to my other hosted website), and an absolutely terrible hack for people who do have access and should be using mod_proxy instead. Is that accurate?
Comment by Señor Pantaloons — November 29, 2005 @ 12:31 pm
I’ve found a fifth method: for every call, dynamically create a new iframe in Javascript (http://developer.apple.com/internet/webcontent/iframe.html). I’m not sure how viable this would be over the long term, but it does work for both Mozilla and IE.
As for a use case, the prime example for me is WSRP, where your portlet is served up from a different server than where your Ajax services are.
Comment by Don Brown — November 29, 2005 @ 1:10 pm
Mr Pants - I wouldn’t say it’s accurate to call this ‘an absolutely terrible hack’, even if you did have the option to use mod_proxy. Well, the mozilla iframe bridge part could be considered terrible, I suppose. But there are cases where this technique is the best option I know, like the LivePage scenario I talked about.
Overall I don’t think any of this goes against the spirit of browser security policies - all I’m trying to do is make XmlHttpRequests work with the same permissions as iframes, which is I don’t think is too much to ask (in fact, many people use iframes for Ajax already). I’m guessing that in the future browsers will relax their standards a bit, and these tricks will no longer be needed.
Comment by Abe — November 29, 2005 @ 1:38 pm
Don - yes, constructing a new iframe for each call is definitely an option. By doing so you’d be able to avoid resorting to hacks like the Mozilla iframe bridge. In the case of LivePage it works out better to keep a single iframe continuously open, though. I’d be interested to see if there are any performance/memory issues with creating and tearing down a lot of iframes.
Comment by Abe — November 29, 2005 @ 1:51 pm
[…] rity, XMLHttpRequest Abe Fettig’s done some important experimenting to arrive at a direct remoting technique, one which […]
Pingback by Software As She’s Developed - Host-Proof Authentication? — November 30, 2005 @ 4:55 am
Does this work across different web services on the same server working on different ports? I have a Rails based Ajax App that needs to talk to a tomcat based set of Ajax services for some specialized work that hasnt been ported over to Ruby on Rails and probably wont. Any ideas?
Comment by Chetan — November 30, 2005 @ 7:52 am
While this approach does seem to at least partially work in current mozillas (I just tested on Mozilla 1.5, windows), it’s far from stable.
Your bridge solution, on named setup, recursively creates new frames for every new call (which has the potential of growing a huge browser memory footprint, once your calls start stacking in larger numbers), and after a while (it seems to be about ten requests), it breaks with the error “Error: Component returned failure code: 0×8000ffff (NS_ERROR_UNEXPECTED) [nsIDOMHTMLIFrameElement.src]
Source file: http://www.fettig.net/playground/ajax-subdomain/test4-bridge.html
Line: 14″.
That is probably a solvable problem. I’m venturing to guess that the reason why this approach does not work in Opera, is that your implementation details rely on (and here is where I actually guess, rather than having done minute research of the domain security standards involved) a hole in Mozilla’s cross site scripting protection, a bug that Opera presumably does not share.
So, in summary, a stable solution to the problem is maybe still in the hands of future tech and standards being in the writing by our friends at the W3C Web APIs Working Group. Hopefully.
On the good (but for most of us rather useless) side, the problem is solvable, at least in Internet Explorer, if given the provisions of adding the site making the xmlhttprequest as “trusted” in local browser security settings. With that, even your first naïve exaple works just fine.
Comment by Johan Sundström — November 30, 2005 @ 10:13 am
Chetan - I believe setting document.domain will work with different port numbers (although maybe not in Opera according to one post I read). I haven’t tested it yet, though.
Comment by Abe — November 30, 2005 @ 10:57 am
[…] 30, 2005 @ 1:16 pm
Firefox 1.5 shipped yesterday, and it brings good news for my work on cross-subdomain XmlH […]
Pingback by Abe Fettig’s Weblog » XmlHttpRequest subdomain update — November 30, 2005 @ 1:16 pm
Johan - The behavior you describe is a bug, not the way it’s designed to work. It only affected Firefox 1.5. And as it turns out, there’s a much better way to do this on FF 1.5. See my latest post.
Comment by Abe — November 30, 2005 @ 1:40 pm
XmlHttpRequest Across Sub-Domains
As you probably know XmlHttpRequest only works if the request is made to the same web server from which the page was fetched (same origin). Here is a way to communicate across sub-domains using XmlHttpRequest without resorting to reverse proxying.
Ab…
Trackback by Simple Thoughts - Java and Web Software — November 30, 2005 @ 1:44 pm
[…] tion works for Internet Explorer and Mozilla groups of browsers. Read about his adventure here. I think such solutions, thoug […]
Pingback by How To Use XmlHttpRequest Across Sub-Domains - Simple Thoughts - Java and Web Software — November 30, 2005 @ 10:48 pm
[…] ecnologia , ajaxian
Abe Fettig, of JotLive and Twisted fame, has written up his experiences trying to get XMLHttpRequest […]
Pingback by www.ienaridens.com » How to make XMLHttpRequest calls to another server in your domain — December 3, 2005 @ 10:06 am
I have a problem.I have 2 servers A and B in same domain.I can not touch server B.I want to access page hosted by B from server A using xmlHttp.How do I do it
Comment by sachin — December 5, 2005 @ 1:00 am
sachin - I don’t know of any way to let server A access data on server B without explicit cooperation from server B. If there was a way, it would probably be considered a security hole. As far as I know you’ll either need to install some pages on server B (to use the technique we’re talking about here) or use a reverse proxy on server A.
Comment by Abe — December 5, 2005 @ 10:35 am
[…] How to make XmlHttpRequest calls to another server in your domain- Interessantíssimo, vale a pena ler. Quando puder irei traduzi-lo […]
Pingback by robson.junior 2.0 » Juntão de Ajax — January 12, 2006 @ 6:23 am
[…] Abe Fettig’s Weblog » How to make XmlHttpRequest calls to another server in your domain […]
Pingback by openlogic.biz » Abe Fettig’s Weblog » How to make XmlHttpRequest calls to another server in your domain — January 30, 2006 @ 9:21 am
[…] This is a good write up on the non-nefarious uses of cross domain Ajax requests, but obviously this has other scary ramifactions. […]
Pingback by ha.ckers.org » Blog Archive » Writeup on cross domain XSS — May 31, 2006 @ 11:45 am
[…] If you don’t already know, there is a limitation in the XmlHttpRequest object used in AJAX applications that won’t allow you to access information across different domains. As Abe Fettig notes: XmlHttpRequest objects are bound by the same origin security policy of browsers, which prevents a page from accessing data from another server. This has put a serious limitation on Ajax developers: you can use XmlHttpRequests to make background calls to a server, but it has to be the same server that served up the current page. Known workarounds for this limitation involve either server-side reverse proxying or bypassing XmlHttpRequest entirely. — fettig.net […]
Pingback by tricklin » Blog Archive » AJAX Parser for Flickr RSS Feeds — June 15, 2006 @ 10:51 am
XMLHttpRequest对象实现的安全性考虑：为什么不能跨域访问…
AJAX通过XMLHttpRequest对象发送http请求，但是只能访问本站点内的资源，站点A的页面通过此对象访问站点B上的资源的行为是被禁止的。原因，参考：…
Trackback by Java newbie — July 23, 2006 @ 11:26 am
[…] How to make XmlHttpRequest calls to another server in your domain […]
Pingback by Do Frames Really Suck? — January 5, 2007 @ 5:40 pm
[…] Abe Fettig’s Weblog » How to make XmlHttpRequest calls to another server in your domain (tags: ajax javascript xmlhttprequest crossdomain security programming howto web development article) […]
Pingback by links for 2007-08-09 « napyfab:blog — August 9, 2007 @ 6:50 pm
[…] Abe Fettig’s Weblog » How to make XmlHttpRequest calls to another server in your domain […]
Pingback by Links for 2007-08-08 [del.icio.us] | programmazione — August 12, 2007 @ 3:39 pm
[…] you are interested in cross-domain scripting, read what some of the experts have done. Such as Abe Fettig. For the other 99.995% of you who don’t give a rats ass, I included a proxy in […]
Pingback by Orbited Blog » Blog Archive » Widespread Proxy Dependence — August 27, 2007 @ 9:46 am