ROT13 Cipher in Assembly

Question

So I've got my ROT13 cipher pretty much doing what I want, however at the end the command prompt shows up on the same line as the last line of output. This is my first project in Assembly so I'm pretty unsure of what I'm doing wrong.

Answer 1

Your program doesn't print a newline at the end of its output, so the cursor is at the end of a non-empty line when it exits. The shell doesn't know this, and prints the next prompt there.

echo foo includes a trailing newline, so when the shell prints the next prompt, the cursor was already at the start of a new line. echo -n foo doesn't include a trailing newline, so it leaves the cursor at the end of a line that starts with foo, and your prompt is tacked on to that, like your program does. Pipe those echo commands into hd to see a hexdump of the ASCII characters they print.

---

So the solution is to make sure your output ends with a newline (ASCII code = 10). You already have that in your msg4: db 10, "Read error", 10 string. (It starts with a newline as well as ending with one.) In C you'd write `"\nRead error\n", but NASM syntax doesn't work that way. It does support C-style escapes inside backquoted strings, but it's typical for people to write newlines with numeric constants.

---

Your user input (that you get from sys_read) should usually end with a newline, unless the user typed 256 characters on a line, or used ctrl-D to make read return early. (Or similarly piped input that doesn't end with a newline, so read hits EOF).

I started to follow the logic of the compares, but it got tiring pretty quickly. I'm not sure what happens to newlines in your input, but I suspect that your code modifies newlines in the buffer. You should probably avoid that, and leave them unchanged. I guess you'd just add them to your list of compares & branch for characters not to modify.

That's probably more useful behaviour for a rot13 program than adding an extra newline to the end of the buffer, or calling sys_write one extra time to print a newline by itself.

---

You can test the system calls your program makes using strace. e.g. strace ./a.out will decode the read() and write() system calls you make.

For more debugging tips, see the bottom of the x86 tag wiki. (Which has lots of useful stuff besides that).

---

BTW, you could do all those cmp al, '?' compares in parallel in an xmm register, with SSE2 (broadcast al to every element of an xmm register, and PCMPEQB with a constant / PMOVMSKB / test/jnz). But don't worry about that until you have a good handle on scalar code.

---

The other way to avoid the rats nest of CMP/JCC would be to white-list alphabetic characters, leaving input characters unmodified by default.

I'm not sure why you only blacklist '1', but not other numbers, or '+' but not '-', and so on.

---

Here's how I'd implement your loop, with some "advanced" tricks to collapse multiple similar conditions into single conditions. See my answer on How to access a char array and change lower case letters to upper case, and vice versa for an explanation of the unsigned-compare trick for isalpha().

;; ROT13 alphabetic characters.  Copy others unmodified.
;;  Untested
L1_top:
    movzx   eax, [esi]              ; get a character
    inc     esi                     ; update source pointer

    mov     edx, eax                ; save a copy of the original
    or      al, 0x20                ; make it lower-case if it's a letter (but we can still detect non-letters after this)

    sub     al, 'a'                 ; chars below 'a' will wrap to a high value
    cmp     al, 'z'-'a'
    ja      .non_alpha              ; jump if the sub wrapped, or the char was greater than 'z'

    ; input char was alphabetic
    sub     dl, 13                  ; modify the original character
    sub     al, 13                  ; check if that takes us out of the alphabet.  Can be a CMP, not SUB if we want.
    jnc     .nocarry
    add     dl, 26                  ; add 26 if the subtract wrapped
    ;add     al, 26                 ; we don't care about the value in al anymore
.nocarry:

    ; dl = the ROT13'ed character, with its original case
.non_alpha:
    mov    [edi], dl
    inc    edi

    dec    ecx                      ; I'm not sure what all the cmp ecx,0  in various branches was for.  Just do it earlier if necessary.
    jnz    L1_top

Originally I was going to actually calculate lower-cased ROT13'ed character in AL, and then find the difference between that and the original lower-cased character, and apply that to DL. But then I realized I could conditionally modify DL in the earlier branching.

    ;; after the or al,0x20:    mov   ah, al       ; don't over-do it with upper-half byte registers.  False dependencies on AMD, and partial-reg merging stalls or slowdowns on pre-Haswell Intel if you're not careful.

    add     al, 'a'                 ; 'a' + al is the lower-cased ROT13 of the input character
    sub     ah, al                  ; ah = lcase(orig) - lcase(rot13)
    sub     dl, ah                  ; apply that delta to the original in dl
    ; dl is the original character - 13 (plus 26 if necessary)