Id like to rephrase the answer:
Whether you choose Flex Viewer or Flex API depends on two things: how customized your application needs to be, and whether you have a customer that needs to be able to configure it without Flash Builder or writing code.
When you make a Flex Viewer app, it will look like a Flex Viewer app, and work mostly like a Flex Viewer app. (As you said, "the standard viewer header, "locked" widget icon locations, skins (the look of the widget windows, etc.), the use of "widgets" to incorporate tools" etc).
The viewer has a 'core', which is around 30 000 line monolith of a code. Modifying this core is very slow: some of the classes have sizes of over thousand lines, and there is not much commentation. This core defines how the Flex Viewer looks and how it essentially works. So if your application needs to be highly customized or cannot look like what Flex Viewer looks like, you probably should do it with just API.
However, it's easy to plug in widgets to the Flex Viewer, and if you settle for how the Viewer feels and looks, you can get very far by just making custom widgets for it. If the ready-made widgets for Viewer satisfy your needs, then go for them. However, if you need highly customized widgets, i recommend making own widgets. The Esri made widget files are often also over 1000 lines with no commentation or architecture in general, and trying to modify them can turn out to be a massive time sink.
Also, be aware that a lot of the component / widget functionality is embedded in the skin files that you can modify by downloading the skins with the API zip, and plugging them into the Viewer.
The Flex Viewer has the benefit, that it's very customizable using .xml-files. E.g. customers and such often wish to accomodate custom-built applications to their map documents personally, but don't have Flash builder and dont want code Flex, and thats what Viewer is well suited for.
Flex dev, Esri Finland