Refactor ConnectJob and TCPConnectJob.

Refactor ConnectJob and TCPConnectJob.
ConnectJob will only call OnConnectJobComplete() on asynchronous completion.
TCPConnectJob now uses DoLoop() internally.
BUG=http://crbug.com/13289
TEST=none

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=19789

[willchan no longer on Chromium, 2009-06-30 22:19:33]

I was about to start writing the late binding code, but then realized that it
indeed was cleaner if I OnConnectJobComplete() only got called in the
asynchronous case, so I went ahead and implemented your suggestion from earlier.
 This also had a small effect on TCPConnectJob, so while I was there, I fixed it
up to use DoLoop() too.  I didn't want the cl to get larger, so I didn't toss in
the late binding stuff here.  I'll do that in a follow up cl.

[eroman, 2009-06-30 23:36:25]

Thanks for making the changes, I think this is much more readable now!

LGTM; I have a couple nits, and one suggestion for refactoring the group's
active count to NOT include the connecting sockets.

http://codereview.chromium.org/151118/diff/1/2
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/1/2#newcode136
Line 136: RemoveActiveSocket(group_name, &group);
What if active socket count was defined to not include the connecting sockets? I
think this would simplify things, since there are a number of calls to
RemoveActiveSocket() here to account for the failure cases.

Here is how it could work:

// Called when a connect job has completed successfully.
void HandOutSocket(Group* group,
                   ClientSocketHandle* handle,
                   ClientSocket* socket,
                   bool is_reused) {
  DCHECK_NE(NULL, socket);
  handle->set_socket(socket);
  handle->set_is_reused(is_reused);
  group->sockets_handed_out_count++;
  group->active_socket_count++;   // <--- we increment here
  CheckSocketCounts(group);
}

Now the body of idle socket loop above becomes simply:

  HandOutSocket(&group, handle, idle_socket.socket, true);

And the synchronous success body becomes simply:

  HandOutSocket(&group, handle, connect_job->ReleaseSocket(), false);




Of course the catch is, that when deciding whether we can "activate a socket"
the bound check becomes:

if (group.active_socket_count + group.connecting_socket_count  ==
max_sockets_per_group_) { ... }

instead of just:

 if (group.active_socket_count == max_sockets_per_group_) {...}

http://codereview.chromium.org/151118/diff/1/2#newcode330
Line 330: RemoveActiveSocket(job->group_name(), &group);
See comment above about not incrementing active count until success. At which
point, it is unnecessary to call this.

http://codereview.chromium.org/151118/diff/1/2#newcode333
Line 333: group.sockets_handed_out_count++;
See comment above; these lines could be replaced by:

  HandOutSocket(&group, handle, job->ReleaseSocket(), false);

http://codereview.chromium.org/151118/diff/1/2#newcode337
Line 337: RemoveConnectJob(handle);
Note if you do the active_count refactor, this would additionally be responsible
for decrementing the connecting_sockets count for the group.

http://codereview.chromium.org/151118/diff/1/2#newcode353
Line 353: const ClientSocketHandle* handle) {
Seems the argument here should be a |ConnectJob* job|

http://codereview.chromium.org/151118/diff/1/3
File net/socket/client_socket_pool_base.h (right):

http://codereview.chromium.org/151118/diff/1/3#newcode53
Line 53: void FillClientSocketHandle(ClientSocketHandle* handle);
How about:

  ClientSocket* ReleaseSocket();

It seems more appropriate to fill the ClientSocketHandle from the pool and keep
this API just about sockets.

The pool would then set the re-used bit, the handed out count, etc...

http://codereview.chromium.org/151118/diff/1/3#newcode60
Line 60: // Begins connecting the socket.  Returns OK on success, ERR_IO_PENDING
if it
Can you explain that the delegate's OnConnectJobComplete() is called in the case
of async completion. And that upon successful completion (whether it be sync or
async), the connected socket can be gotten at through ReleaseSocket() (or
FillClientSocketHandle(), whatever this ends up being named).

[willchan no longer on Chromium, 2009-06-30 23:53:51]

+phajdan.jr

Heads up for Pawel since using different counters will give him merge conflicts.
 I'm implementing the suggested changed now.

[Paweł Hajdan Jr., 2009-06-30 23:56:09]

Thanks for notice. Feel free to check in. I have various presentations and
orientations this week, so it will take me a while to fix review issues (write
tests!) for my patch.

[willchan no longer on Chromium, 2009-07-01 00:35:42]

Running the try jobs now...

http://codereview.chromium.org/151118/diff/1/2
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/1/2#newcode136
Line 136: RemoveActiveSocket(group_name, &group);
On 2009/06/30 23:36:25, eroman wrote:
> What if active socket count was defined to not include the connecting sockets?
I
> think this would simplify things, since there are a number of calls to
> RemoveActiveSocket() here to account for the failure cases.
> 
> Here is how it could work:
> 
> // Called when a connect job has completed successfully.
> void HandOutSocket(Group* group,
>                    ClientSocketHandle* handle,
>                    ClientSocket* socket,
>                    bool is_reused) {
>   DCHECK_NE(NULL, socket);
>   handle->set_socket(socket);
>   handle->set_is_reused(is_reused);
>   group->sockets_handed_out_count++;
>   group->active_socket_count++;   // <--- we increment here
>   CheckSocketCounts(group);
> }
> 
> Now the body of idle socket loop above becomes simply:
> 
>   HandOutSocket(&group, handle, idle_socket.socket, true);
> 
> And the synchronous success body becomes simply:
> 
>   HandOutSocket(&group, handle, connect_job->ReleaseSocket(), false);
> 
> 
> 
> 
> Of course the catch is, that when deciding whether we can "activate a socket"
> the bound check becomes:
> 
> if (group.active_socket_count + group.connecting_socket_count  ==
> max_sockets_per_group_) { ... }
> 
> instead of just:
> 
>  if (group.active_socket_count == max_sockets_per_group_) {...}

Done.

http://codereview.chromium.org/151118/diff/1/2#newcode330
Line 330: RemoveActiveSocket(job->group_name(), &group);
On 2009/06/30 23:36:25, eroman wrote:
> See comment above about not incrementing active count until success. At which
> point, it is unnecessary to call this.

It's still necessary.  We might need to call ProcessPendingRequests().  Also, we
might need to delete the Group from |group_map_| if connecting_requests.empty().

http://codereview.chromium.org/151118/diff/1/2#newcode333
Line 333: group.sockets_handed_out_count++;
On 2009/06/30 23:36:25, eroman wrote:
> See comment above; these lines could be replaced by:
> 
>   HandOutSocket(&group, handle, job->ReleaseSocket(), false);

Done.

http://codereview.chromium.org/151118/diff/1/2#newcode337
Line 337: RemoveConnectJob(handle);
On 2009/06/30 23:36:25, eroman wrote:
> Note if you do the active_count refactor, this would additionally be
responsible
> for decrementing the connecting_sockets count for the group.

What count?  I'm not maintaining any separate counter, I'm just using the size()
of the stl container.

http://codereview.chromium.org/151118/diff/1/2#newcode353
Line 353: const ClientSocketHandle* handle) {
On 2009/06/30 23:36:25, eroman wrote:
> Seems the argument here should be a |ConnectJob* job|

When late binding is implemented, then yes.  Right now, CancelRequest() needs to
be able to remove a job via |handle|.  CancelRequest() will be a no-op for
connecting requests when we have late binding.

http://codereview.chromium.org/151118/diff/1/3
File net/socket/client_socket_pool_base.h (right):

http://codereview.chromium.org/151118/diff/1/3#newcode53
Line 53: void FillClientSocketHandle(ClientSocketHandle* handle);
On 2009/06/30 23:36:25, eroman wrote:
> How about:
> 
>   ClientSocket* ReleaseSocket();
> 
> It seems more appropriate to fill the ClientSocketHandle from the pool and
keep
> this API just about sockets.
> 
> The pool would then set the re-used bit, the handed out count, etc...

Done.

[eroman, 2009-07-01 01:06:43]

http://codereview.chromium.org/151118/diff/9/1016
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/9/1016#newcode125
Line 125: RemoveActiveSocket(group_name, &group);
The use of RemoveActiveSocket() is odd.

First off it doesn't remove the active socket (or even decrement the active
count), so it should be named something more like "RemovedActiveSocket" or
"DidRemoveActiveSocket".

Secondly, since the connecting sockets are no longer called "active" sockets, it
doesn't feel right to call RemoveActiveSocket() for it.

Can't you move this logic into ProcessPendingQueue/OnConnectJobComplete(), and
make RemoveActiveSocket() actually decrement the active count?

http://codereview.chromium.org/151118/diff/9/1016#newcode247
Line 247: group.connecting_requests.empty()) {
How about making this a group.empty() method ?

http://codereview.chromium.org/151118/diff/9/1016#newcode312
Line 312: RemoveActiveSocket(group_name, &group);
See earlier comment.

This naming is strange, since when it comes to "active_count", connecting jobs
are not considered "active sockets".

Why not just process the queue after having invoked the callback?

(also note that not deleting empty groups isn't a big deal, since the idle timer
will eventually get it anyway.)

http://codereview.chromium.org/151118/diff/9/1016#newcode346
Line 346: int rv = RequestSocket(
It seems that RequestSocket() is calling "RemoveActiveSocket()" just so this
callsite works in case of synchronous completion.

Why not check for synchronous completion here, and call ProcessPendingRequest()
again if necessary.

This also gives the ability to invoke the callback before processing the queue,
so things complete in the right order.

[willchan no longer on Chromium, 2009-07-01 02:16:52]

http://codereview.chromium.org/151118/diff/9/1016
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/9/1016#newcode125
Line 125: RemoveActiveSocket(group_name, &group);
On 2009/07/01 01:06:43, eroman wrote:
> The use of RemoveActiveSocket() is odd.
> 
> First off it doesn't remove the active socket (or even decrement the active
> count), so it should be named something more like "RemovedActiveSocket" or
> "DidRemoveActiveSocket".
> 
> Secondly, since the connecting sockets are no longer called "active" sockets,
it
> doesn't feel right to call RemoveActiveSocket() for it.
> 
> Can't you move this logic into ProcessPendingQueue/OnConnectJobComplete(), and
> make RemoveActiveSocket() actually decrement the active count?

We only need to decrement the active socket count in DoReleaseSocket().  I've
renamed RemoveActiveSocket.

http://codereview.chromium.org/151118/diff/9/1016#newcode247
Line 247: group.connecting_requests.empty()) {
On 2009/07/01 01:06:43, eroman wrote:
> How about making this a group.empty() method ?

Done.

http://codereview.chromium.org/151118/diff/9/1016#newcode312
Line 312: RemoveActiveSocket(group_name, &group);
On 2009/07/01 01:06:43, eroman wrote:
> See earlier comment.
> 
> This naming is strange, since when it comes to "active_count", connecting jobs
> are not considered "active sockets".
> 
> Why not just process the queue after having invoked the callback?
> 
> (also note that not deleting empty groups isn't a big deal, since the idle
timer
> will eventually get it anyway.)

I've renamed it.  I'd like to delete the empty groups since it'll expose the
bugs faster if there are any, and also it prevents a very small edge case where
possibly group_map_ grows unbounded if we never have any idle sockets.

http://codereview.chromium.org/151118/diff/9/1016#newcode346
Line 346: int rv = RequestSocket(
On 2009/07/01 01:06:43, eroman wrote:
> It seems that RequestSocket() is calling "RemoveActiveSocket()" just so this
> callsite works in case of synchronous completion.
> 
> Why not check for synchronous completion here, and call
ProcessPendingRequest()
> again if necessary.
> 
> This also gives the ability to invoke the callback before processing the
queue,
> so things complete in the right order.

Done.

[eroman, 2009-07-01 04:11:44]

Can you add a unit-test that hits the new dispatch order (i.e. when a queued
request completes synchronously during the async completion of the first
request) ?

http://codereview.chromium.org/151118/diff/20/1022
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/20/1022#newcode311
Line 311: OnAvailableSocketSlot(group_name, &group);
I'm not sure that it is safe to access &group here.

Couldn't running the callback have erased the group?

Maybe OnAvailableSocketSlot(const std::string&, Group*) and
ProcessPendingRequest(const std::string&, Group*) could be combined into
something like

  ProcessGroup(const std::string& group_name);

Which would lookup the group, delete empty entries, and kick off  a new request
if there is an available slot.


What if the callback deletes |this|.... is that a possibility?

http://codereview.chromium.org/151118/diff/20/1022#newcode351
Line 351: GroupMap::iterator it = group_map_.find(group_name);
See comment above, seems like these 2 call-sites should share the same solution.

http://codereview.chromium.org/151118/diff/20/1023
File net/socket/client_socket_pool_base.h (right):

http://codereview.chromium.org/151118/diff/20/1023#newcode223
Line 223: // Sees if we should use the socket slot.  Might delete the Group from
I would remove the first phrase, it doesn't add much.

[willchan no longer on Chromium, 2009-07-01 05:18:47]

I ended adding two tests on accident.  The one that actually covers the case you
asked about is on the bottom.  I added another that gives us more coverage, but
it wasn't actually what you asked for :P

http://codereview.chromium.org/151118/diff/20/1022
File net/socket/client_socket_pool_base.cc (right):

http://codereview.chromium.org/151118/diff/20/1022#newcode311
Line 311: OnAvailableSocketSlot(group_name, &group);
On 2009/07/01 04:11:44, eroman wrote:
> I'm not sure that it is safe to access &group here.
> 
> Couldn't running the callback have erased the group?
> 
> Maybe OnAvailableSocketSlot(const std::string&, Group*) and
> ProcessPendingRequest(const std::string&, Group*) could be combined into
> something like
> 
>   ProcessGroup(const std::string& group_name);
> 
> Which would lookup the group, delete empty entries, and kick off  a new
request
> if there is an available slot.

I thought about this, and perhaps I'm prematurely optimizing, but doing so would
require extra unnecessary hashes into group_map_, since we'd always do lookups
even though we don't necessarily need to.  I added a
MaybeOnAvailableSocketSlot() method to handle this case.

> 
> 
> What if the callback deletes |this|.... is that a possibility?

I...don't know.  I'm not familiar with the shutdown process.  That's the only
time in which it could happen, because the URLRequestContext will maintain a
reference to the pools, so it shouldn't be possible to go away until the
URLRequestContext releases its reference.  If that is not the final reference to
the ClientSocketPoolBase, then it might be able to have the callback ending up
deleting |this|.

If this is actually an issue, I think we can solve it with a
ScopedRunnableMethodFactory by posting a task instead of synchronously calling
MaybeOnAvailableSocketSlot().  What do you think?

http://codereview.chromium.org/151118/diff/20/1022#newcode351
Line 351: GroupMap::iterator it = group_map_.find(group_name);
On 2009/07/01 04:11:44, eroman wrote:
> See comment above, seems like these 2 call-sites should share the same
solution.

Done.

[eroman, 2009-07-01 22:19:02]

lgtm

http://codereview.chromium.org/151118/diff/24/1036
File net/socket/client_socket_pool_base_unittest.cc (right):

http://codereview.chromium.org/151118/diff/24/1036#newcode672
Line 672: EXPECT_EQ(&req3, request_order_[1]);
Why does this finish before request 2?

Is it because request 1 finishes before request 2, therefore request 3 which is
sync gets started from request 1's completion, therfore finishes before request
2?

Whatever the case, please add a comment explaining this expectation.

[willchan no longer on Chromium, 2009-07-01 22:28:00]

Thanks for the thorough review!

http://codereview.chromium.org/151118/diff/24/1036
File net/socket/client_socket_pool_base_unittest.cc (right):

http://codereview.chromium.org/151118/diff/24/1036#newcode672
Line 672: EXPECT_EQ(&req3, request_order_[1]);
On 2009/07/01 22:19:03, eroman wrote:
> Why does this finish before request 2?
> 
> Is it because request 1 finishes before request 2, therefore request 3 which
is
> sync gets started from request 1's completion, therfore finishes before
request
> 2?
> 
> Whatever the case, please add a comment explaining this expectation.

Correct.  I've updated the comments.