Yes, but if you can find a chip with public documentation, it's a well-defined initialization sequence.
I learned assembly on TI's AM335x "Sitara" series and it was great, mostly because of the BeagleBone- it has onboard JTAG-over-USB, meaning you can single-step your way through your bare-metal code, set breakpoints, etc.
Yes, that gets the CPU running - but for practical work you usually need to do some board-specific setup like configuring clocks and turning on the DRAM.
For an Cortex M0 the clocks default to something sane. And RAM is static. One of my projects the reset vector is just the address of init function written in C. That does nothing more than copy the data section from flash to ram and call _start()
There is a bunch of peripheral set up but it can be done from C.