I probably learned most of my assembly from trying to reverse engineer Outpost 2. I had a little bit of prior knowledge, but not much experience before I started on that.
There were a few ways that I learned. First, you probably want to try a little bit, just to get your feet wet. And maybe stumble around getting error messages to find out that you can't do certain things, that might otherwise seem obvious to do. (Try MOV [addr1], [addr2]).
Next a spent a lot of time reading assembly to see what was done and how it was done. This was largely reading Outpost 2 code. Some of it was just reading code that was hand crafted in assembly. Perhaps some speed sensitive code, or some processor specific feature, or CPUID type of code.
After that, I found I was trying to reverse engineer some of these things into a higher level language such as C/C++. Initially it was just the general concept, such as rewriting the algorithm so it would do the same thing. Then after a while I started wondering more about exactness. Not just an algorithm that would do the same thing, but one that was written in such a way that when compiled it would produce the same or similar assembly code. You'll sometimes find yourself puzzling over certain instruction sequences, and how they came to be. Sometimes things will compile to much less efficient code than you might have written yourself if starting from assembly. You'll definately start learning something about compilers at about that point.
There is also writing simple programs and compiling them to see what sort of assembly is produced. Try changing the source code in minor ways. See what extra declarative keywords do. (__stdcall, __cdecl, virtual, volatile, containment vs. inheritance of classes) Definately instructive as to how compilers work. You'll begin to understand why those keywords and there a lot better, and also how to use them properly.
You can also learn how the assembly mnemonics map to machine code. (Think of the actual binary image/hex byte dump, as opposed to text based source code). Then you'll really start to understand certain things about why assembly is the way it is. Like why you can't do MOV [addr1], [addr2]. (Assuming x86 of course). You also get to learn about certain efficiency aspects, like using 16 bit values in 32 bit code. (Those instructions need an operand size prefix, which adds to the instruction length). There are also weird effects you might not expect, like changing the register used for the same instruciton might lead to a different sized encoding, or a completely different encoding altogether. You'll also get to see how the architecture has changed over the years (legacy encodings, vs newer more "orthogonal" encodings). You can also see how the same instruction mnemonic is found in different places throughout the opcode tables for the different operand formats it supports (such as JMP and CALL).
By this point, you might consider trying to write a few "assembly" programs directly in machine code. Try writing a simple "Hello World" program by writing raw bytes to disk with a hex editor, or for extra fun using:
The "CON" stands for console. This will copy what is entered at the keyboard directly into the file. The .com files are binary executables with no header, so they'd be much easier to write this way than an exe file, which has a rather large and fairly complex header (at least if you're trying to write one by hand). There are of course certain restrictions on the values you can enter that way, as some keys are processed specially, such as backspace. You can use the Alt+Numpad trick to enter in all the non restricted values, which gets you wroughly 240? of the 256 possible bytes you can enter. I believe Ctrl+Z denotes end of file (completes the copy) under DOS? It might be Ctrl+D or something under Unix.
You can also write a program to format the boot sector of the harddrive this way. I saw an example of this online somewhere. Probably won't work under Windows though, so unless you're using DOS, and the BIOS isn't set to protect the boot sector, this probably won't do anything.
After you've done that, you can go on to writing a hex editor to make future work easier.
Oh, and at some point, you'll want to try writing a few serious algorithms for yourself without reference assembly code to guide you.
If that's still not enough to satisfy you, you can try poking around with different architectures. Perhaps there's an old NES game that you liked? Well, FCEUltra is an emulator with a nice built in debugger. Just grab a book or online article on 6502 assembly, and away you go. (I recommend the book, as there is no doubt an ancient copy collecting dust on a public library shelf somewhere, and it probably of higher value than many online documents). I've heard the SNES also uses a processor from the same line, and in many ways, the CPU was backwards compatible (although the supporting hardware likely wasn't). And just to horrify all you old nintendo lovers, which all too often have rather strong feelings about a certain line of computers, guess what else the 6502 was used in?