host always wants more

So, what is the problem ? Is it requesting the same data (exactly the same request arguments) ? The host may use a probe-first strategy to get potentially large descriptors (eg a concatenated CONFIG descriptor), so a descriptor might be requested two times - the first time with a minimum wLength just to get the exact length, the second time with the exact length, which is now known, to get the whole thing.
Have you check what is going on on the host side with a software analyzer ?

TRNIF
BD0_IN
TOKEN_IN
In Token EP0
USB_dev_req=
255
ZLP
Last Packet

- Why are you sending that ZLP ???
 
- Where are the OUT tokens ?
 
LE:

__John
No. Is there a software analyzer that will show what the host is doing during enumeration?

Yes and no. Software analyzers are only usable after a certain point (I think only after the 2nd GET_DESCRIPTOR request), so the initial part must work or you will not see anything in there. They cannot replace device-side logging if you are building from scratch, but later on they can be useful in upper layers because you can see some aspects from the host's perspective.
 

I think you might have misunderstood the way that the PIC SIE works. You do not get notifications on received tokens, but rather on completed transactions ! 
As far as IN transactions are concerned, the SIE signals when it has successfully sent a packet that you have previously queued in the corresponding BD. It does not mean that you received an IN token that requires more data, it means that the SIE has receieved an IN token and it has already successfully responded with a DATAx packet that you have previously passed to the SIE via the BD. Whether or not you need to continue queuing more data for the next IN token is up to you to decide (eg you track transfer total length and the length that you have already transferred, or whatever ...).

Thanks bytencoder, you have given me much to think about.

SETUP and OUT tokens both use the same BD (the output BD for the particular endpoint). You must discern between them via the BDxSTAT<5:2>.
 

bytencoder

__John
Ok, I do arm ep0 out, I do it after receiving any setup token but before processing the token.

How come ? You need to do this after you process the request (in case of SETUP token).
Because only then you can decide how to arm :

- Request success
- No data stage ->  expect next SETUP (arm normally, towards an EP0 default buffer)
- Data stage, dir from device ->  expect Status OUT / aborting SETUP (arm normally, towards an EP0 default buffer)
- Data stage, dir from host ->  expect first Data OUT / aborting SETUP (arm normally, towards destination of Data)
- Request failure -> protocol stall (arm with BSTALL, towards an EP0 default buffer)

Before or after. It does not matter. The fact is there is enough information in the setup packet to determine how EP0 OUT needs to be setup. However I do not believe it has been correctly implemented  with this:
 
 BD0_OUT.CNT = 8;
 BD0_OUT.STAT = (!(USB_host_setup_package.bmRequestType & 0x80) && (USB_host_setup_package.wLengthLO || USB_host_setup_package.wLengthHI)) ? (DTSEN | UOWN | DTS) : (DTSEN | UOWN);
 
This translates to:
Out packet with data? (DTSEN | UOWN | DTS) else (DTSEN | UOWN)
 
Therefore IN packets with data do not receive the OUT STATUS ZLP because the OUT endpoint is not set correctly (DTSEN | UOWN)  DATA0 is wrong DATA1 is correct. This accounts for the missing OUT STATUS stage.
 
Any DATA stage for either IN or OUT transfers requires that the OUT endpoint be set to DATA1 immediately after the setup packet.
 
Therefore (so far) the correct code should be:
 
 BD0_OUT.CNT = 8;
 BD0_OUT.STAT = (USB_host_setup_package.wLengthLO || USB_host_setup_package.wLengthHI) ? (DTSEN | UOWN | DTS) : (DTSEN | UOWN);
 
However this still is not correct because the HOST may abort the data stage transfer at any stage and issue a new setup packet (data0). To allow for either a DATA0 or DATA1 OUT/SETUP packet the data toggle sync should be disabled if we have any data stage. We can leave it enabled when we know that there is no data stage as only a SETUP packet can come next.
 
 BD0_OUT.CNT = 8;
  BD0_OUT.STAT = (USB_host_setup_package.wLengthLO || USB_host_setup_package.wLengthHI) ? (UOWN) : (DTSEN | UOWN);
 
Now we can receive either OUT or SETUP packets correctly.
 
I know that the system the OP used is also used on at least three other USB stacks. The original microchip assembler stack for the 16C745 and the stacks based on this code, Brad Minch and dangerous prototypes. Still it is not correct and one thing they all seem to have in common is that they need to send a ZLP IN after the descriptors are fully sent. This is not correct and I wonder if the two issues are related? Certainly when I corrected the above error on my version the need for a ZLP was removed.
 
The "most" correct thing to do IMHO when the host asks for more IN data when there isn't any is to return a protocol stall on the next IN transaction and OUT for good measure. (Remembering that the STALL is ignored in the case of a SETUP packet arriving, Microchip use this loophole in their stack). 
 
However, (I believe but will double check at some stage) technically not returning the same amount of data as requested violates the USB spec that does not allow for a device to return less EP0 data than requested. The truth of the matter is that OSes force us to do this by asking for stupid amounts of IN data (in wLength of the setup packet) even AFTER it knows the length of the descriptors. Windows is particularly bad at this.
 
Anyway it is not the fault of the firmware that the OS does stupid things. Us firmware writers simply have to code around it and I think the most correct solution is what I have outline here. But at 1:45AM and after 12 hours of coding I reserve the right to be wrong  :)
 
 
 
 
 
 
 

newfound:

The original microchip assembler stack for the 16C745 and the stacks based on this code, Brad Minch and dangerous prototypes. Still it is not correct and one thing they all seem to have in common is that they need to send a ZLP IN after the descriptors are fully sent. This is not correct and I wonder if the two issues are related? Certainly when I corrected the above error on my version the need for a ZLP was removed.

On the DATA stage of Control Read transfer, ZLP is required when,
1) the actural transfer length is less than requesed by wLength of SETUP data
AND
2) the actural transfer length is just the multiple of wMaxPacket0 of EP0

I examined Brad Minch's Lab 3 ( http://pe.ece.olin.edu/ece/projects/lab3c.zip ).
The problem of Lab 3 is that It follows just above 2), and it doesn't count in 1)

Tsuneo

A simplified answer to your first question is,

When data toggle mismatch occurs, host still returns ACK to the transaction, but host discards the payload data. And then, host put another IN transaction.
On the firmware side, the last transaction successfully completes, even if data toggle (DTS) would be wrong. But the firmware sees another IN transaction, after this IN transaction.
You are seeing this extra IN transaction caused by data toggle mismatch.
 
Tsuneo

bytencoder:

Question: The data toggle mechanism specifies that when uncorrupted data with invalid toggle arrives, the receiver should accept it (send ACK handshake), but discards it, on the presumption that the relevant data has already been received.

The USB spec (usb_20.pdf) supposes Corrupted ACK as the cause of data toggle mismatch.

8.6 Data Toggle Synchronization and Retry
8.6.4 Corrupted ACK Handshake
The transmitter is the last and only agent to know for sure whether a transaction has been successful, due to its receiving an ACK handshake. A lost or corrupted ACK handshake can lead to a temporary loss of synchronization between transmitter and receiver as shown in Figure 8-47. Here the transmitter issues a valid data packet, which is successfully acquired by the receiver; however, the ACK handshake is corrupted.

But how can this be true in the case when there is a mismatch on the very first transaction ? How current host implementations handle this case ?

If host would receive data toggle mismatch at the first transaction, it means the device firmware does not properly initialize data toggle. There isn't any chance that the mismatch of the first transaction could occur by noise on the line.

The response of PC host controller (hardware) to mismatched data toggle is same, regardless of the first or latter transactions, ACK, discard and retry.

Tsuneo

But I don't see how the transfer can ever be successful; using faux ZLP like it was mentioned above cannot really fix it, it makes matter worse, as the packet from the first transaction is discarded but the transfer succeeds !

I'm not sure, faux ZLP actually plays any active role on his problem.
On his log, the second and third Get_Descriptors seem to finish with timeout of DATA stage.
And then, no STATUS stage is seen.

Tsuneo