SitePoint Sponsor

User Tag List

Results 1 to 4 of 4

Hybrid View

  1. #1
    SitePoint Member
    Join Date
    Jul 2013
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Mysterious UTF8 Encoding Problem

    I have a website running on App Engine and I store my data in Datastore. If the text contains Turkish letters, they are replaced with ? in a diamond. But interestingly, sometimes I edit them and they appear fine or while they appear correct they start to contain <?> again.
    I don't know when it happens and when it doesn't. So I change something in the code, then it works, but later I see those <?> again. I call it mysterious because of that and I don't know how to explain better. I will try again if it's not clear.

    It is 2013 and I am struggling with characters :/ By the way, I am not expert on web development, I am learning while making this. So maybe I forgot a very simple thing.
    Please create a record here to see (I can delete them later or you can delete afterwards)

    http://petkayit.appspot.com/app

    These are the turkish characters for copy pasting: ş ı ğ
    Thanks

  2. #2
    SitePoint Member
    Join Date
    Jul 2013
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I sort of re generate the error.
    If I edit all the fields in the form leaving no strange char of <?>, the turkish letters sent to server are replaced to <?>
    but if I leave one <?> in the form and re-write the rest, then it saves what is seen exactly.
    looks like my browser or something tries to guess the encoding and converts into another encoding.
    I check in the server right after parsing json, and they are already replaced.
    Actually I should check also before json.Unmarshal. I'll try now

  3. #3
    SitePoint Member
    Join Date
    Jul 2013
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Still solutionless. I compared the request headers in inspector window for two form submissions and they are not much different. how can I debug more?

  4. #4
    SitePoint Member
    Join Date
    Jul 2013
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Figured what causes it but it is not related to Javascript or JQuery. Looks like it is a bug in Appengine's ParseUpload method and there is a discussion already here if anyone has the same trouble they can look here:
    https://code.google.com/p/googleappe...detail?id=9420
    My silly work around is to send a <?> which is String.fromCharCode(65533) with the form data.
    It is also interesting that if I type "ş ı ğ " it worked fine but when I write a meaningful sentence it guess the context (my opinion). So I didn't understand when it happens or not.


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •