Saturday, May 3, 2014

WinUSB communication with STM32 round 2 (Part 1: Firmware)

Introduction


I did another WinUsb device. This time I've used STM32F4. Project in Google Code repository is for STM32F4DISCOVERY board featuring STM32F407VGT6.
I wanted to make it so that it wouldn't need Windows side driver information in a form of .inf file. I did it with Windows OS USB descriptors. They are described in subsection on MS "Dev Center - Hardware" Drivers page (links at the end of post). 



Result of device providing these descriptors is Windows loading one of MS provided drivers and associating the device with it as instructed in descriptor gotten from device when plugged in USB port.
"MS provided drivers" are not to be mistaken with "USB device class drivers included in Windows" which are drivers for external HDDs, webcams, HIDs...
WinUSB is one of MS provided drivers providing raw communication with device, e.g. WriteBuffer/ReadBuffer. This means that PC side application and firmware in device must follow some defined communication protocol. These are well defined for standard class USB devices like mouse and keyboard or external HDD but with WinUSB custom protocol can/must be defined. It can be quite simple like adding 4 byte header to transferred buffer holding the length of buffer.

[Update] Windows enumeration

While preparing PC side demo application to demonstrate communication I found out that categorization differs for single interface devices and composite multiple interface devices. Image above shows 2 devices in  device manager each plugged in own USB port and each with single USB interface associated with WinUSB driver. So does the image below with additional Properties window.
Single device with single WinUSB driver associated interface. 

But notice how composite device (having multiple USB interfaces) is shown on next image. Two WinUSB  associated interfaces were reported in descriptors by device:
Single device with two WinUSB driver associated interfaces.
Three devices are shown in Device Manager. One Composite device and two WinUSB devices - one for each interface. WinUSB devices have Composite device set as parent. Note how "Bus reported device description" differs for single interface device from dual interface device.
To test both configuration un/comment WINUSBCOMM_DUAL_INTERFACE_TEST macro in usbd_WinUSBComm.h file.

Windows OS USB descriptors


Text below refers to 'interfaces'. Term is used for a set of endpoints/pipes used in a group. Interfaces have own descriptors.

MS OS String Descriptor


When device is plugged in USB port for the first time on a Windows machine it also receives a standard USB string descriptor request requesting the string descriptor with index 0xEE among other requests during enumeration. For first few indices device responds with standard string descriptors which are Manufacturer, Product and Serial strings (also Language IDs for index 0). Device can ignore 0xEE string descriptor request or respond with data predefined by MS. If so Windows will then query for additional MS specific descriptors:
  • Extended Compat ID OS Descriptor
  • Extended Properties OS Descriptor
  • others not relevant here
The content of MS OS String Descriptor must be:

Value Field Size in Bytes Description
0x12,
bLength
1
Length of the descriptor
0x03,
bDescriptorType
1
Descriptor type - string
0x4D, 0x00,
0x53, 0x00,
0x46, 0x00,
0x54, 0x00,
0x31, 0x00,
0x30, 0x00,
0x30, 0x00,
qwSignature
14
MSFT100” Signature field
MS_VendorCode,
bMS_VendorCode
1
Vendor-specific Vendor code
0x00
bPad
1
Pad field

MS_VendorCode is a value (defined as custom macro in project) which Windows will use to form requests for other descriptors. This gives device developer some leeway to add OS descriptors functionality without clashing with other custom USB requests which device might already support. For new and simple USB devices supporting only single WinUSB interface the vendor code is not that much relevant. Just make it non zero so that it won't cause problems on USB 3 hosts (see specification for more).
Note that Windows queries this descriptor only once. It can be a hassle during development. Information that OS descriptors have been queried for some device is stored in registry under
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\usbflags\VVVVPPPPRRRR
(VVVV - vendor ID; PPPP - product ID; RRRR - revision).
Delete VVVVPPPPRRRR key and also uninstall the device with utility like USDDeview to always get fresh device plug in behavior.

Extended Compat ID OS Descriptor


After device responds with proper 'MS OS String Descriptor' Windows sends a setup packet with request for 'Compat ID OS Descriptor'. Format of request is as follows:

Setup request field Value
bmRequestType b11000000 (device to host transfer direction; vendor type request; recipient is device)
bRequest
MS_VendorCode
wValue 0x0000
wIndex
0x0004 (request for an extended compat ID OS descriptor)
wLength
0x0010 or full length of descriptor

This a chance for device to inform Windows which driver should be loaded. Content below shows WinUSB driver association:

Value Field Size in Bytes Description
0x28, 0x00, 0x00, 0x00,
dwLength
4
The length, in bytes, of the complete extended compat ID descriptor
0x00, 0x01,
bcdVersion
2
The descriptor’s version number, in binary coded decimal (BCD) format
0x04, 0x00,
wIndex
2
An index that identifies the particular OS feature descriptor
0x01,
bCount
1
The number of custom property sections
0, 0, 0, 0, 0, 0, 0,
RESERVED
7
Reserved
0,
bFirstInterfaceNumber
1
The interface or function number
0,
RESERVED
1
Reserved
0x57, 0x49, 0x4E, 0x55, 0x53, 0x42, 0x00, 0x00,
compatibleID
8
The function’s compatible ID ("WINUSB")
0, 0, 0, 0, 0, 0, 0, 0,
subCompatibleID
8
The function’s subcompatible ID (none)
0, 0, 0, 0, 0, 0,
RESERVED
6
Reserved

Descriptor is queried twice. First with only header length (0x10) in wLength field. Then, after full length is known to Windows (from dwLength field) with full length in wLength field.
Each interface in device can have own association. Multiple sections follow the header (in yellow) in case of more interfaces. Descriptor is queried on device level and all associations must be listed in single descriptor. bFirstInterfaceNumber denotes the index of associated interface in each section.

Extended Properties OS Descriptor


After associating the driver with device Windows queries for interface properties. These are meant to provide information which is usually provided in .inf files of drivers, e.g.:

[USB_Install.HW]
AddReg=Dev_AddReg

[Dev_AddReg]
HKR,,DeviceInterfaceGUIDs,0x10000,"{EA0BD5C3-50F3-4888-84B4-74E50E1649DB}"

Properties Descriptor request is also issued twice (or more for descriptors larger than 64kByte). First with wLength set to 0x0A to only retrieve the header and later with wLength set to full length gotten from dwLength.
Following is the layout of USB Setup request:

Setup request field Value
bmRequestType b11000001 (device to host transfer direction; vendor type request; recipient is interface)
bRequest
MS_VendorCode
wValue 0xppii (pp – PageNumber for descriptors larger than 64kB; ii – interface index)
wIndex
0x0005 (request is for an extended properties OS descriptor)
wLength
0x000A or full length of descriptor

Note that properties are queried per interface. So if your device has multiple interfaces the request is send at least twice for each interface (BTW the number of interfaces is known from standard USB configuration descriptor). Following descriptors defines two properties:

  • DeviceInterfaceGUID
    This identifies interface with GUID which is used later in Windows application when querying the OS for devices with SetupDiGetClassDevs
  • Label
    This should be a friendly name displayed for device in Device manager. But apparently it only works form Windows 8 on

Value Field Size in Bytes Description
0xCC, 0x00, 0x00, 0x00,
dwLength
4
The length, in bytes, of the complete extended properties descriptor
0x00, 0x01,
bcdVersion
2
The descriptor’s version number, in binary coded decimal (BCD) format
0x05, 0x00,
wIndex
2
The index for extended properties OS descriptors
0x02, 0x00,
wCount
2
The number of custom property sections that follow the header section
0x84, 0x00, 0x00, 0x00,
dwSize
4
The size of this custom properties section
0x01, 0x00, 0x00, 0x00,
dwPropertyDataType
4
Property data format
0x28, 0x00,
wPropertyNameLength
2
Property name length (PNL)
'D',0, 'e',0, 'v',0, 'i',0, 'c',0, 'e',0, 'I',0, 'n',0, 't',0, 'e',0, 'r',0, 'f',0, 'a',0, 'c',0, 'e',0, 'G',0, 'U',0, 'I',0, 'D',0, 0, 0,
bPropertyName
PNL
The property name (DeviceInterfaceGUID)
0x4E, 0x00, 0x00, 0x00,
dwPropertyDataLength
4
Length of the buffer holding the property data (PDL)
'{',0, 'E',0, 'A',0, '0',0, 'B',0, 'D',0, '5',0, 'C',0, '3',0, '-',0, '5',0, '0',0, 'F',0, '3',0, '-',0, '4',0, '8',0, '8',0, '8',0, '-',0, '8',0, '4',0, 'B',0, '4',0,
'-',0, '7',0, '4',0, 'E',0, '5',0, '0',0, 'E',0, '1',0, '6',0, '4',0, '9',0, 'D',0, 'B',0, '}',0, 0, 0,
bPropertyData
PDL
PDL Format-dependent Property data ({EA0BD5C3-50F3-4888-84B4-74E50E1649DB})
0x3E, 0x00, 0x00, 0x00,
dwSize
4
The size of this custom properties section
0x01, 0x00, 0x00, 0x00,
dwPropertyDataType
4
Property data format
0x0C, 0x00,
wPropertyNameLength
2
Property name length (PNL)
'L',0, 'a',0, 'b',0, 'e',0, 'l',0, 0, 0,
bPropertyName
PNL
The property name (Label)
0x24, 0x00, 0x00, 0x00,
dwPropertyDataLength
4
Length of the buffer holding the property data (PDL)
'W',0, 'i',0, 'n',0, 'U',0, 'S',0, 'B',0, 'C',0, 'o',0, 'm',0, 'm',0, ' ',0, 'd',0, 'e',0, 'v',0, 'i',0, 'c',0, 'e',0, 0, 0

bPropertyData
PDL
PDL Format-dependent Property data (WinUSBComm Device)
These properties are stored in Windows registry at:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USB\VID_xxxx&PID_xxxx\sssssssss\Device Parameters
Use USDDeview or similar utility to uninstall device and delete this entry while developing.

HW Info


To supply the board from OTG USB micro AB connector you need to connect PA9 pin to 5V pin. This is possible because VBUS is (strangely) connected directly to PA9 VBUS sensing input.

[Update] Comm library

Communication is implemented by implementing Comm layer so that it can be used with Comm library explained separately.

This is first post in this series (of total 2 posts). See 2.

Reference


Links tree below is for my future reference. You probably want to see descriptor specification (in bold).



27 comments:

  1. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. What do you mean with "Winusb template of Visual Studio"?

      You are asking quite widely. I can't answer shortly. As you can see the project is a bit complex and spread across few posts. But nevertheless, I assume you want raw communication from your Windows application to microcontroller. WinUSB is useful because you don't need to write your own kernel driver for device. Optionally you can make your device respond properly to OS descriptor requests and that eliminates the need for driver INF files which need to be signed with Code Signing certificate.
      So, for your device to be recognized as WinUSB device (by Windows) you need to associate it with WinUSB driver. You can do that by writing INF file telling Windows that a device with your VID and PID uses WinUSB.sys. Or you respond to Extended Compat ID OS Descriptor request with "WINUSB" string in compatibleID field as described in the post above.

      If you are asking how you can detect your WinUSB device from application, that is done through DeviceInterfaceGUID property in Extended Properties OS Descriptor. Just be sure to match GUID from descriptor with one used in SetupDiGetClassDevs call.

      Delete
  2. Hello!
    First of all let me thank you for sharing your knowledge!

    I quite noob with programming uC units this low level, but i have an stm32f407vg board that i would like to learn how to use.
    I´m trying to make a program that will generate a series of clocks to control a device and read that device output with 8 different GPIOs. Stack those bytes and send them via USB in FS mode to the PC.
    I´m trying to deal with the USB and came across your blog.
    I struggled a bit to assemble your project and libs, but i think i´ve almost done it.
    I´m getting the following error: ../src/main.c:147: undefined reference to `_sram'
    Where the variable _sram is the argument in this function, i think;
    WinUSBCommSTM32F4_Init(&asCommLayers[0], &sSTM32F4USB.m_sWinUSBCommSTM32F4, &_sram, 0x8FF0);
    _sram is declared like this: extern unsigned char _sram;

    I don´t really understand what this error means.

    Can you give me any help?
    Thanks

    ReplyDelete
    Replies
    1. Hi,

      _sram is just a symbol (not an actual variable) defined in linker file, known at link time. Only it's address is used (&_sram). It is set in linker file as "_sram = ORIGIN(SRAM);" But to use it in source it must somehow be known at compile time. That is why it is declared as "extern unsigned char _sram;" This is saying to compiler: "Yo, somewhere is a variable _sram of type unsigned char." and when compiler hits it's usage with address operator '&' it says: "Ah, an address of some variable is needed here, I'll let linker deal with it." This is a trick to get link time values in your code.

      SRAM is used as communication buffer. WinUSBCommSTM32F4_Init takes the address and size of the buffer. You can use some other address and size if you wish (I think it needs to be in SRAM though but I'm not sure anymore).

      Also, see the STM32F4xxxG.ind and GCCFLASH_CCM.ind included in STM32F407xG.ind and how it is set as ${LinkerFilePath} in Eclipse project C/C++Build/Build Variables and later used in C/C++ Build/Settings/Tool Settings/Cross GCC Linker/Miscellaneous.

      Delete
    2. Hello.
      Thanks for your answer!
      Hum...you gave me some stuff to look into.
      I´m using AC6 to program the board. This IDE already has the settings for the stm32f4Discovery board. It is Eclipse based.
      What i did was to start an empty project coupled with HAL libs and carefully place your files in it.
      It already starts with a LinkerScript.ld which i don´t really know what it does.
      But now that i looked a little into it, i see the resemblance in the structure of the script and those files you told me to look in.
      I copied the content of your files to the linkerScript. I see _sram = ORIGIN(SRAM);
      Also i copied this line: SRAM(RWX) : ORIGIN = 0x20000000, LENGTH = 0x20000 /* 128k */
      and now the MEMORY command looks like this:
      MEMORY
      {
      FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 1024K
      RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 192K
      SRAM (RWX) : ORIGIN = 0x20000000, LENGTH = 0x20000 /* 128k */
      MEMORY_B1 (rx) : ORIGIN = 0x60000000, LENGTH = 0K
      }

      No build errors shown! =)
      Now..i´ll move to the windows side of the program!!
      Thanks a lot for your help!

      Delete
    3. NP.

      Be careful! Since I don't know the rest of your linker file I can't say for sure but you have now defined two memory regions on same address. It might be that something else is put/used in your RAM region. BTW, you could achieve same result by only adding _sram = ORIGIN(RAM); to your linker file.

      Delete
    4. Hey!
      Right. Strange things did happen. I would flash the uC and see the instruction pointer on the debugger pointing to the start of the program, i would press the run button and it would go about its business.
      If i would press the reset button on the board, it would run the previous program i burned.
      That behavior was probably due to those changes in the memory.
      But i just picked the *.bin file generated and burned it with the ST-Link utility - problem solved...(i think)
      I´m still struggling with the VS program.
      Maybe some file is missing. Pulling the files from the GoogleCode was hardworking because i didn´t find a way other than "save link as" to every file. The "checkout" thing really didn´t do it for me.
      I´m getting a lot of missing references and stuff. But i´ll work my way around, i hope.

      Delete
    5. Strange. Last time I checkeddd the projects built. I know Google is discontinuing the code hosting. I didn't have the time to move code elsewhere. But I could checkout complete repo with Tortoise from http://found-bits.googlecode.com/svn/trunk/.

      Regarding strange reset, check 32bit value at address 0x00000004 after downloading the code. Value must point to your reset routine and usually changes when downloading new version of FW due to code position changes. I doubt it's because of linker file change - does empty Hello World example behave the same?
      Check in linker file what goes in RAM. It would be a scope of sections ending with ">RAM" (without quotes). I think your linker file puts variables there and uses it for stack. It's not OK to use it for communication buffer.

      Checkout my other post for linker file explanation.

      Delete
    6. Hello!
      Yep, i was unaware of the Tortoise part. When i googled for "google checkout" all i got was that it was closed and stuff about google wallet, etc..
      Now i got the dir structure right off the link. I´m not yet into all those project, and code management applications and platforms.

      The VS project gives me errors with ToolsVersion not being the right one. I installed VS2010 and still the same error.
      I changed the option "platfrom Toolset" on the property page of the project to "inherit from parent or project defaults"
      Now the error looks like this:
      Project file contains ToolsVersion="12.0". This toolset is unknown or missing. You may be able to resolve this by installing the appropriate .NET Framework for this toolset. Treating the project as if it had ToolsVersion="4.0".
      stdafx.cpp
      c:\found-bits\sources\projects\libs\comm\commtest\winclient\stdafx.h(37): fatal error C1083: Cannot open include file: 'afxcontrolbars.h': No such file or directory

      And now that i used the USBDeview i see that the program in the uC is not running properly, because no USB device is detected other than the debugger port.

      Can you help me?

      Delete
    7. Hi,

      Well start digging. It works on my PC :). First install VS2013 community edition, it's free - you need to register after a month (also free).

      What do you mean debug port only? Are you using this board? FW uses second USB port.

      Delete
    8. Yes! That´s the board. The MINI USB is always plugged, and allows programming and debugging, right? And the MICRO USB should be recognized like a windows USB device.
      But the connection (in any form) is not recognized. It is like if it´s not even plugged.
      I have VS2008,2010 and 2012 installed. I´ve been messing with configurations and now the projects don´t even open any more. "C:\Miscrosoft.Cpp.Default.props was not found"...Moving to VS2013.
      Also I installed .netFW 4.5, buildtools2013 and WinDriverKit 7.1.0. to fix errors that keep appearing..

      How much time did you spent coding?
      I´ve been reading your post about Linker. Interesting stuff.
      Also i found some differences between a linkerscript.ld in some other project that runs ok, and the project that has your code - many sections of the script end up with FLASH in the ok version and in RAM in your version. Even after changing that the program does not seem to work.
      I think my gaps in knowledge wont let me get this to work any time soon...

      Delete
    9. Don't complicate with Visual Studio. 2013 is all you need.

      You might be looking at my linker file for RAM GCCRAM_CCM.ind. It is not used in the project. It's there just as reference for how to make a project that executes from RAM - faster download, no flash programming needed, for tests only. GCCFLASH_CCM.ind is used in the project.

      Can't really tell how much I spent coding this as it was a part of some other project. This is stripped version of that with USB stuff only for my future reference. Also, I had prior knowledge of low level USB and STM32 USB periherals. I did spend few days to study WinUSB, though.

      Don't give up! Keep at it and you will eventually got through.

      Delete
    10. That usb analyzer is quite nice, it let´s us see the protocol in action. Kind of wireshark.
      Also i installed VS2013 and all i needed to do was to build. No errors....go figure...
      Made me look for a pc with win8. Fortunately i was able to get one...
      The part 2 of the windows program is to create a main that accesses the classes you´ve made.
      That´s gonna take a while. So i´ll turn my attention to the uC.
      Thanks a lot for your help. I´ll let you know if i make any progress...or regression :P

      Delete
    11. Info : dropped 'gdb' connection
      Info : accepting 'gdb' connection on tcp/3333
      target state: halted
      target halted due to debug-request, current mode: Thread
      xPSR: 0x01000000 pc: 0x080001dc msp: 0x1ffffff8
      target state: halted
      target halted due to debug-request, current mode: Thread
      xPSR: 0x01000000 pc: 0x080001dc msp: 0x1ffffff8
      target state: halted
      target halted due to breakpoint, current mode: Thread
      xPSR: 0x61000000 pc: 0x20000042 msp: 0x1ffffff8
      target state: halted
      target halted due to debug-request, current mode: Thread
      xPSR: 0x01000000 pc: 0x080001dc msp: 0x1ffffff8
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000008)
      Error: address + size wrapped(0xffffffff, 0x00000004)
      Error: address + size wrapped(0xffffffff, 0x00000004)

      Delete
    12. Ufff, I don't recognize these. I'm using external debugger. See how to hook it up on my other post.

      Delete
    13. I posted something before the error. It had the 1st part of the error and some other stuff, but it´s not showing on the blog. Never mind...
      I asked you if you could recommend me some specific reading about programming uC.
      I´ve been reading about the different parts of memory on the stm23f4.
      You use a lot of entries to CCM block. And i don´t know what that is.

      Delete
    14. Hmmm, my blog :). Nothing really comes to mind. I know there were some short guides some 10 years ago from Atmel and Microchip. I learned through projects. Doing stuff on assembler level forces you to get to know architecture. Reading uC reference manuals...

      CCM is Closely Coupled Memory in STM32F4. Only for read and write, not for fetching. Checkout RM and data sheet.

      Delete
    15. Yep!
      I´m always saying to the teachers "what about teaching some assembler", and they´r like "oh, that´s not used any more"..and i have tried to follow some tutorials about it. But always have to leave them half way trough because it is to apart of all the things that i need to do and i fail to the relation..Oh well...coming from an arduino platform to this kind of uC is quite a shock..
      But maybe this will kick start me into some heavy ARM stuff, like what you do.
      Thanks a lot for all the info and help. Keep on posting!

      Delete
    16. Hello! How´s it going?
      I manage to get a USB connection. One of the problems was that my usbd_conf.c file didn´t have a #define for usbFS.
      Nevertheless i´m ending up in the default handler while inside the function USBD_LL_Start.
      The struct sent to this function, pdev, is really big, but right at the beginning it has "id" and it´s content is 0. At least that´s what the debugger tell´s me.
      Is this normal?
      The USB connections is detected by windows but ends up with an unrecognized device...

      Delete
    17. That's too low, I don't know by heart. And I don't have time to investigate this for you.
      BTW, I suggest you check Eclipse project for compiler command line defines (-D).

      Delete
  3. Poing!
    One define was missing (HSE=8000000)....it´s working now! I already spent half an hour crying of happiness.
    But i´m still to put your windows program to work to actually see any action.
    Thanks!

    ReplyDelete
  4. Hello!Can i get a copy of your project source code,which i couldn't find at the google code webside.Thanks a lot.And my email:oberli@126.com

    ReplyDelete
    Replies
    1. Hi. I didn't find time to migrate the code to GitHub yet. But there is an archive download still available on Google Code site: https://code.google.com/archive/p/found-bits/source/default/source

      Delete
    2. Hi.I have received your email.Thank you for your kindness.Thanks a lot.

      Delete
  5. Hello!
    Thanks for an important information. Wold it be worth trying software such as this USB Capture (http://www.eltima.com/products/usb-capture/) to log USB communication?

    ReplyDelete
    Replies
    1. Probably. I have used FREE USB Protocol Analyzer from HHD Software and tried USBTrace from sysnucleus. AFAIK these are all software hook loggers. They attach to USB hooks prepared by in Windows through some API I forgot. Log they produce is quite helpful.

      Delete