1. ## RC4 encryption problem

Hello,

I'm trying to fix an issue with a classic asp website that uses Mike Shaffer's RC4 algorithm for encryption and decryption:

Code ASP:
```Dim sbox(255)
Dim key(255)

Sub RC4Initialize(strPwd)
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
':::  This routine called by EnDeCrypt function. Initializes the :::
':::  sbox and the key array)                                    :::
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

dim tempSwap
dim a, b, intLength

intLength = len(strPwd)
For a = 0 To 255
key(a) = asc(mid(strpwd, (a mod intLength)+1, 1))
sbox(a) = a
next

b = 0
For a = 0 To 255
b = (b + sbox(a) + key(a)) Mod 256
tempSwap = sbox(a)
sbox(a) = sbox(b)
sbox(b) = tempSwap
Next

End Sub

Function EnDeCrypt(plaintxt, psw)
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
':::  This routine does all the work. Call it both to ENcrypt    :::
':::  and to DEcrypt your data.                                  :::
':::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

dim temp
dim a, i, j, k
dim cipherby
dim cipher

i = 0
j = 0

RC4Initialize psw

For a = 1 To Len(plaintxt)
i = (i + 1) Mod 256
j = (j + sbox(i)) Mod 256
temp = sbox(i)
sbox(i) = sbox(j)
sbox(j) = temp

k = sbox((sbox(i) + sbox(j)) Mod 256)

cipherby = Asc(Mid(plaintxt, a, 1)) Xor k
cipher = cipher & Chr(cipherby)
Next

EnDeCrypt = cipher

End Function```

I'm basically just passing in a 15 digit card number. Recently, the encryption method seems to fail whilst looping through each character in the EnDeCrypt function. The value returned (cipher) appears to be made up of 15 characters, but some of these are html entities of the encrypted character...not the characters themselves.

It seems to fall down when the value returned by the following line (cipherby) returns 0 (zero):
cipherby = Asc(Mid(plaintxt, a, 1)) Xor k

In other words, the next line, Chr(0), is returning a Null value, which causes all sorts of issues when the returned cipher is fed into an INSERT query.

I have also noticed that the characterset/page encoding affects the encryption of the characters, so I'm just wondering if I'm using the wrong characterset (utf-8) or codebase perhaps...

Any help is appreciated.

Thanks

2. Looks like this code is intended for single-byte characters sets. So if you use a double-byte character set it will likely throw a wobbly. My guess is that this is expecting a Western Latin-1 type character set.

3. If I change the charset to ISO-8859-1 it doesn't appear to make much difference to the problem.

Incidentally and for my understanding, can double-byte character sets contain characters in a single-byte representation? i.e. < 256. Because the encryption sometimes does work, which would seem to indicate that in those situations the encryption algorithm is churning out acceptable characters.

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•