We recently had a fun little issue where a seemingly random set of users with varying combinations of Windows 2000, XP, and Vista as well as Office 2003 and Office 2007 started having issues editing documents from SharePoint documents libraries.
This didn’t effect all our users, which made it trickier to debug, but when we were finally able to do a Fiddler trace it appeared that the calls failing were all going to the /_vti_bin/author.dll library. This is a long-standing holdover from the bad-old days of FrontPage and FrontPage extensions that still gets used in several scenarios by Office clients (including SharePoint Designer).
The RPC error we kept getting was “Form submission could not be processed because it exceeded the maximum length”. This seemed really odd since the actual request size in these traces was only a few bytes. We knew our Maximum Upload Size value was bigger than a few bytes, having just gone through an exercise to increase it to 200MB.
I’ll just cut to the chase rather than bore you with all the dead ends we went down (Google was unfortunately not a lot of help on this one). It turns out someone had changed our Maximum Upload Size to 2048MB. I don’t know why this happened, but it’s slightly bigger than the maximum size noted in the stsadm docs for the max-file-post-size (2047MB max). If you spend a lot of time with powers of 2, you’ll probably notice this is suspiciously close to one of the magic numbers that tends to result in things like overflow errors (i.e. it’s right at the maximum size of a signed 32bit integer). My guess is that the old FrontPage RPC stuff still uses 32-bit integers to store this kind of thing, and it probably “rolled over” to look like a negative number. So any calls to author.dll were “too large” no matter how big they actually were. I don’t know for sure, but it seems like a high probability.
Anyhow, just change the Max Upload Size back to a more reasonable value resolved the issue and our Office clients were happily RPC-ing away again.
