Issue Details (XML | Word | Printable)

Key: MULE-3666
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Dan Diephouse
Reporter: Bill Graham
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Mule

MuleReceiverServlet endpoint object leak causes stack overflow - sample webapp fatally broken

Created: 12/Sep/08 07:20 PM   Updated: 01/Oct/08 04:19 PM
Component/s: Transport: HTTP/ HTTPS
Affects Version/s: 2.0.2
Fix Version/s: 2.0 ITR7, 2.1.0

Time Tracking:
Not Specified

File Attachments: 1. Text File MuleReceiverServlet.java.patch.txt (2 kB)
2. Text File MuleReceiverServlet.java.patch.txt (1 kB)


Labels:
User impact: High


 Description  « Hide
Every time a request is made to MuleReceiverServlet the endpoint bound to the receiver gets re-wrapped. Over time this causes a stack overflow. You should to reproduce on the sample webapp with this script:

for ((i=0;i<100000;i++)); do curl -d "payload=foo" http://c18-rb-dev40-app1:7006/mule-examples/rest?endpoint=greeter; done > /dev/null 2>&1

Here's an excerpt of mine from the mailing list describing the details of the problem:

To follow up on this I've found the bug in the Mule code, but I'm not sure what the correct fix is. The MuleReceiverServlet has this line that gets executed on every request at line 315:

receiver.setEndpoint(new DynamicURIInboundEndpoint(receiver.getEndpoint(), new MuleEndpointURI(
getRequestUrl(httpServletRequest))));

The receiver has the same lifecycle as the servlet, so it gets re-used for each request. The problem is that for each request, the receivers endpoint basically gets wrapped one more time. In our tests after about 20K requests we get a stack overflow exception, because there are 20K DynamicURIInboundEndpoints wrapping each other.

It can be changed to only wrap the endpoint if it's not already an instance of DynamicURIInboundEndpoint, but I don't know if that would meet intent of the author of this code.

Can anyone with knowledge of this code point out what the proper fix is? This bug will cause Mule to constistantly crash.

----- Original Message ----
From: Bill Graham <emailbillgraham@yahoo.com>
To: Mule users <user@mule.codehaus.org>
Sent: Friday, September 12, 2008 10:37:11 AM
Subject: [mule-user] Infinite exception loop at DynamicURIInboundEndpoint

Hi,

This morning I found an exception that was in the logs that was caught in an infinite loop. Is this a known issue, or has anyone seen this? Unfortunately it's not entirely clear what request triggered this, nor how to reproduce, but we're investigating that. This is was in the logs:

[2008-09-11 19:13:40.353] at java.util.AbstractList.iterator(AbstractList.java:337)
[2008-09-11 19:13:40.353] at java.util.AbstractList.hashCode(AbstractList.java:627)
[2008-09-11 19:13:40.353] at java.util.Collections$UnmodifiableList.hashCode(Collections.java:1153)
[2008-09-11 19:13:40.353] at org.mule.util.ClassUtils.hash(ClassUtils.java:788)
[2008-09-11 19:13:40.353] at org.mule.endpoint.AbstractEndpoint.hashCode(AbstractEndpoint.java:315)
[2008-09-11 19:13:40.353] at org.mule.endpoint.DynamicURIInboundEndpoint.hashCode(DynamicURIInboundEndpoint.java:170)
[2008-09-11 19:13:40.353] at org.mule.endpoint.DynamicURIInboundEndpoint.hashCode(DynamicURIInboundEndpoint.java:170)
[2008-09-11 19:13:40.353] at org.mule.endpoint.DynamicURIInboundEndpoint.hashCode(DynamicURIInboundEndpoint.java:170)
[2008-09-11 19:13:40.353] at org.mule.endpoint.DynamicURIInboundEndpoint.hashCode(DynamicURIInboundEndpoint.java:170)

[[ repeats indefinitely ]]

Looking at the code (Mule 2.0.2) DynamicURIInboundEndpoint wraps another endpoint, and hashCode traverses through calling each. In my case it seems the DynamicURIInboundEndpoint is wrapping itself. Any ideas how that could happen? Here's the code from DynamicURIInboundEndpoint with line 170 pointed out:

public int hashCode()

{ final int prime = 31; int result = 1; result = prime * result + ((dynamicEndpointURI == null) ? 0 : dynamicEndpointURI.hashCode()); result = prime * result + ((endpoint == null) ? 0 : endpoint.hashCode()); <<---- line 170 return result; }

thanks,
Bill



 All   Comments   Work Log   Change History   Transitions   FishEye      Sort Order: Ascending order - Click to sort in descending order
Bill Graham added a comment - 15/Sep/08 12:03 PM
Attached is a patch for this fix. The diff was made from the 2.0.x branch and the change is not committed:
mule/branches/mule-2.0.x/transports/servlet/src/main/java/org/mule/transport/servlet]$ svn diff MuleReceiverServlet.java > MuleReceiverServlet.java.patch.txt

Let me know if I should open a new 'patch' JIRA.


Bill Graham added a comment - 15/Sep/08 12:13 PM
Also above my script was set to test against my host, which instead should be locahost:

for ((i=0;i<100000;i++)); do curl -d "payload=foo" http://localhost:8080/mule-examples/rest?endpoint=greeter; done > /dev/null 2>&1


Bill Graham added a comment - 15/Sep/08 12:52 PM
That last attachment was missing an import statement - please disregard it. [[embarrassed]]

Ken Yagen added a comment - 30/Sep/08 05:52 PM
Assumed to be the same issue as EE-762 so flagging for the iteration.

Dan Diephouse added a comment - 01/Oct/08 04:19 PM
Thanks for the patch Bill!

This is now fixed here: http://svn.mule.codehaus.org/changelog/mule/?cs=12828 - and it will be in the 2.1 release.