c18n: Set curthread->tcb to real tcb upon thread start

[dpgao]

When a pthread is created, RTLD stores in its tcb field a fake tcb-like structure that contains a capability to the actual tcb and a capability to the thread's stack lookup table. This is done so that when the thread is entered, it does not need to create a stack lookup table, which involves a memory allocation that cannot be performed without a stack lookup table in the first place.

This commit fixes a mistake where the tcb field is not restored to the real tcb upon thread entry.

[jrtc27]

If rtld is screwing with curthread, why doesn’t it fix it up itself?

[dpgao]

If rtld is screwing with curthread, why doesn’t it fix it up itself?

It certainly can. But right now RTLD is not aware of the internal structure of struct pthread. I'm not sure if it's a good idea to teach it about that.

[jrtc27]

Then how does it store the fake value there in the first place?

[dpgao]

Then how does it store the fake value there in the first place?

In _thr_alloc the value returned by _tcb_ctor is stored in the tcb field. _tcb_ctor calls into RTLD to create the (fake) tcb.

[bsdjhb]

I would suggest rewording the commit message a bit which I think would be less confusing (I think it led to Jess's question), maybe something like:

c18n: Set curthread->tcb to real tcb upon entry

When a pthread is created, the TCB is allocated in _tcb_ctor by calling _rtld_allocate_tls().  In the case of c18n, RTLD allocates both a real TCB
and a fake tcb-like structure that contains a pointer to the real TCB and
a pointer to the new thread's stack lookup table.  This is done so that when
the thread is entered, it does not need to create a stack lookup table, which
involves a memory allocation that cannot be performed without a stack
lookup table in the first place.

This commit fixes a mistake where the TCB field was still pointing to the fake
tcb-like structure instead of the real TCB after thread entry.

A few points: 1) A restore operation is usually paired with some sort of prior save, so using restore here is confusing since the field was never previously set to the real TCB. Using "Set" is clearer. 2) This message still doesn't really explain to me why the TCB has to be wrong initially. The comment in the commit seems to suggest that we are not allocating the real TCB during _tcb_ctor() (as my proposed log above suggests) but perhaps instead the real TCB is allocated later when the new thread starts running? In particular, I don't understand why you have to have the fake TCB to allocate the stack table. Why can't you just allocate the stack table and real TCB using _rtld_allocate_tls() and still return a pointer to the real TCB from that function?

[dpgao]

In particular, I don't understand why you have to have the fake TCB to allocate the stack table. Why can't you just allocate the stack table and real TCB using _rtld_allocate_tls() and still return a pointer to the real TCB from that function?

The reason is that if I allocate the stack table in _rtld_allocate_tls() but just return a pointer to the real TCB, there would seem to be no way for me to retrieve the stack lookup table upon thread entry, so I have to embed a capability to the stack lookup table in the parameters passed to the kernel when I create a new thread.

[bsdjhb]

In particular, I don't understand why you have to have the fake TCB to allocate the stack table. Why can't you just allocate the stack table and real TCB using _rtld_allocate_tls() and still return a pointer to the real TCB from that function?

The reason is that if I allocate the stack table in _rtld_allocate_tls() but just return a pointer to the real TCB, there would seem to be no way for me to retrieve the stack lookup table upon thread entry, so I have to embed a capability to the stack lookup table in the parameters passed to the kernel when I create a new thread.

In particular, I don't understand why you have to have the fake TCB to allocate the stack table. Why can't you just allocate the stack table and real TCB using _rtld_allocate_tls() and still return a pointer to the real TCB from that function?

The reason is that if I allocate the stack table in _rtld_allocate_tls() but just return a pointer to the real TCB, there would seem to be no way for me to retrieve the stack lookup table upon thread entry, so I have to embed a capability to the stack lookup table in the parameters passed to the kernel when I create a new thread.

That would be a useful detail in the commit log then. :) Also, there should be a comment to that effect in c18n_allocate_tcb that cross references _rtld_thread_start, maybe something like:

/*
 * Create a fake wrapper tcb containing pointers to the real TCB
 * and stack table.  This is passed to the new thread as its initial
 * TCB.  Once the thread starts executing in _rtld_thread_start
 * it will set the thread's TCB to the real TCB after initializing its
 * stack table.
 */