Removing unwanted characters in HTML Editor

[javascript7]

I have an issue when copy and pasting into the Quill Editor from another source (ie word or an email)

I am seeing 3 question marks that remain in the database even though the editor is blank.

When I look at the database, this still remains:

<p class="ql-align-center"><strong class="ql-size-large" style="background-color: rgb(255, 255, 255); color: rgb(32, 33, 36);"><span class="ql-cursor">???</span></strong></p>

In doing some research, I saw the below at this link:

entering a newline with bold or italics results in a <span class="ql-cursor></span> with a unicode character inside the span, potentially causing character encoding problems with databases

elementValue = elementValue.replace(/**[^<]***</span>/g, "");

I am not sure how to implement this and would appreciate help in this area.

I am using ASP Classic and the database field is named “comments”.

This is what I currently use:

Dim regEx
Set regEx = New RegExp
regEx.Global = true
regEx.IgnoreCase = True
regEx.Pattern = "\s{2,}"
comments = Trim(regEx.Replace(comments, " "))

Any ideas to best remove or prevent this would be appreciated. It seems like the only issue that I have is when i use copy and paste. No issues from straight typing.

Thank you

[m_hutley]

Could at least figure out where its going wrong by reading the string as a byte array and visually seeing where/what the odd character(s) are.

[javascript7]

@m_hutley

visually seeing where/what the odd character(s) are

Thanks for your reply, I appreciate it.

Could you explain more what you mean by that?

The editor works fine except for when pasting text from Word. This is an editor that @James_Hibbard helped me with, who did an exceptional job, so I don’t think it has to do with any bad coding. It just rather normal to do that. “???” is a common string of characters. Do you how I can add that to the regEx

@javascript7

regEx.Pattern = "\s{2,}"

This is the part I really don’t understand.

[m_hutley]

I dont think the ??? is actually three question marks, it’s a representation of a character it couldnt render.

Try this.

let utf8Encode = new TextEncoder();
let arrayout = utf8Encode.encode("yourpastedstringhere");
console.log(arrayout);

And look to see if any of the bytes are odd numbers (something less than 65 or greater than 122)

[javascript7]

@m_hutley

it’s a representation of a character it couldnt render.

That is very logical and makes sense.

@m_hutley

let utf8Encode = new TextEncoder();
let arrayout = utf8Encode.encode("yourpastedstringhere");
console.log(arrayout);

How to I arrange or handle is from the code that I first submitted?

@javascript7

Dim regEx
Set regEx = New RegExp
regEx.Global = true
regEx.IgnoreCase = True
regEx.Pattern = "\s{2,}"
comments = Trim(regEx.Replace(comments, " "))

“comments” is the pasted string but I need to get this all in the database.

[javascript7]

@m_hutley

let arrayout = utf8Encode.encode("yourpastedstringhere");

I believe you have something there. In doing more research, I saw this on a website, which is what you are saying.

Microsoft’s products are all Unicode compliant. It doesn’t make sense that you have to change your locale to fix the issue.

The ??? indicates that Unicode or UTF-8 is not being recognized properly (rather than being misdiagnosed as a different charset (perhaps between the program and the clipboard).

So what I did was to take a Word document and type a word and then made it bold. I copied to clipboard and then pasted it into the html editor.

Once I did that the ??? came up again. So it’s definitely a MS Word copy/paste deal.

If you think there is a work around that I appreciate it.

[James_Hibbard]

Hi,

You should look at the clipboard module.

Clipboard Module - Quill

You should be able to add a matcher which will cause the editor to paste plain text only:

quill.clipboard.addMatcher (Node.ELEMENT_NODE, function (node, delta) {
  var plaintext = $(node).text ();
  return new Delta().insert (plaintext);
});

I found that here:

How to paste plain text in a Quill-based editor

The accepted answer is 6 years old, so I’m not sure if the jQuery dependency is still a thing.

[javascript7]

Thanks for that, I will look into that. “How to paste plain text in a Quill-based editor” I will look at the article but pasting “plain text” is not the issue. It’s when pasting a MS Word, let’s say its the word “hello” and I bold it and then paste into the editor. I believe that is not plain text because it’s carrying over the formatting of MS Word and causing those question marks.

PS - edit
I should of looked at this first from the link you gave.

[code]

quill.clipboard.addMatcher (Node.ELEMENT_NODE, function (node, delta) {
    var plaintext = $ (node).text ();
    return new Delta().insert (plaintext);
});
```[/code]

I will give this a try and report back.  I get it - it's removing the formatting.

[James_Hibbard]

@javascript7

I should of looked at this first from the link you gave.

Yup What you need to do is to hook into the “onpaste” event in the editor, then strip out any undesired formatting from Word and co. before the text is pasted into the editor.

[javascript7]

@James_Hibbard

What you need to do is to hook into the “onpaste” event in the editor

In looking at the code, that script is not on the page, it looks like it’s from this source from the page:

<script src="https://cdn.quilljs.com/2.0.0-dev.2/quill.js"></script>

This is what I found. Is this what you are talking about?

this.onPaste(e, range);\n    }\n    e.preventDefault();\n  }\n\n  onCopy(e, range) {\n    const text = this.quill.getText(range);\n    const html = this.quill.getSemanticHTML(range);\n    e.clipboardData.setData('text/plain', text);\n    e.clipboardData.setData('text/html', html);\n  }\n\n  onPaste(e, range) {\n    const html = e.clipboardData.getData('text/html');\n    const text = e.clipboardData.getData('text/plain');\n    const pastedDelta = this.convert({ text, html });\n    debug.log('onPaste', pastedDelta, { text, html });\n    const delta = new _quillDelta2.default().retain(range.index).delete(range.length).concat(pastedDelta);\n    this.quill.updateContents(delta, _quill2.default.sources.USER);\n    // range.length contributes to delta.length()\n    this.quill.setSelection(delta.length() - range.length, _quill2.default.sources.SILENT);\n 

@James_Hibbard

You should be able to add a matcher which will cause the editor to paste plain text only:

Not familiar with “matcher”. When you add a match, does that mean you take out the old code and insert this new matcher code? Or can you just place the matcher code on the page and does it override the other code?

[James_Hibbard]

@javascript7

<script src="https://cdn.quilljs.com/2.0.0-dev.2/quill.js"></script>

That’s the Quill library.

@javascript7

Not familiar with “matcher”.

Me neither, really. I just did a bit of searching and that was what came up.

Nonetheless, here’s something of an explanation:

Quill has a concept of modules, which allow Quill’s behavior and functionality to be customized. The Clipboard module is required by Quill and does not need to be included explicitly.

The Clipboard handles copy, cut and paste between Quill and external applications. This has an API which allows us to customize this behaviour. We can do this with matchers and the addMatcher method.

// Initialize Quill
const quill = new Quill('#editor', {
  ...
}

// Add a matcher
quill.cliboard.addMatcher(selector, (node, delta) => {
  // Code to customize the clipboard behavior here
});

For a description of what matchers, nodes and deltas are, please refer to the Clipboard module’s docs (linked above).

By following the docs and examples I found on Stack Overflow, I was able to get the following matcher to work for me and to strip out any formatting on paste:

quill.clipboard.addMatcher (Node.ELEMENT_NODE, (node, delta) => {
  const plaintext = node.innerText;
  const Delta = Quill.import('delta');
  return new Delta().insert(plaintext);
});

Here is the complete code so that you can run it in context. I took the setup code from Quill’s quick start page.

<!DOCTYPE html>
<html lang="en" >
<head>
  <meta charset="utf-8">
  <title>Quill clipboard demo</title>
  <link href="https://cdn.quilljs.com/1.3.6/quill.snow.css" rel="stylesheet">
</head>
<body>

  <div id="editor">
    <p>Hello World!</p>
    <p>Some initial <strong>bold</strong> text</p>
    <p><br></p>
  </div>

  <script src="https://cdn.quilljs.com/1.3.6/quill.js"></script>
  <script>
    const quill = new Quill('#editor', {
      theme: 'snow',
    });

    quill.clipboard.addMatcher (Node.ELEMENT_NODE, (node, delta) => {
      const plaintext = node.innerText;
      const Delta = Quill.import('delta');
      return new Delta().insert(plaintext);
    });
  </script>
</body>
</html>

To see the demo in action try copy/pasting any formatted text (from this thread, for example). With the matcher active, it should be inserted as plain text. If you delete the matcher, it should retain its formatting.

[javascript7]

Here’s that part of the code that I ended up with:

<script src="https://cdn.quilljs.com/2.0.0-dev.2/quill.js"></script>

    <script>
        // Fetch content from database here
        var editorContent = '<%=rs("comments")%>';
        var editor = document.getElementById('editor-container');
        var hiddenInput = document.getElementById('myHtml');
        var form = document.forms.mainform;

        var quill = new Quill(editor, {
            modules: {
                table: true,
                syntax: true,
                toolbar: '#toolbar-container'
            },
            placeholder: 'You have lots of space to tell your story!.',
            theme: 'snow'
        });

        var table = quill.getModule('table');

        document.querySelector('#insert-table').addEventListener('click', function () {
            table.insertTable(2, 2);
        });
        document.querySelector('#insert-row-above').addEventListener('click', function () {
            table.insertRowAbove();
        });
        document.querySelector('#insert-row-below').addEventListener('click', function () {
            table.insertRowBelow();
        });
        document.querySelector('#insert-column-left').addEventListener('click', function () {
            table.insertColumnLeft();
        });
        document.querySelector('#insert-column-right').addEventListener('click', function () {
            table.insertColumnRight();
        });
        document.querySelector('#delete-row').addEventListener('click', function () {
            table.deleteRow();
        });
        document.querySelector('#delete-column').addEventListener('click', function () {
            table.deleteColumn();
        });
        document.querySelector('#delete-table').addEventListener('click', function () {
            table.deleteTable();
        });

        editor.firstChild.innerHTML = editorContent;

        form.addEventListener('submit', function (e) {
            e.preventDefault();
            hiddenInput.value = editor.firstChild.innerHTML;
            this.submit();
        });

        function selectLocalImage() {
            var input = document.createElement('input');
            input.setAttribute('type', 'file');
            input.onchange = function () {
                var file = input.files[0]; // file type is only image.
				
                if (/^image\//.test(file.type)) {
                    saveToServer(file);
                } else {
                    console.warn('You could only upload images.');
                }
            };
			input.click(); // Listen upload local image and save to server
        }

        function saveToServer(file) {
            var fd = new FormData();
            fd.append('image', file);
            var xhr = new XMLHttpRequest();
            xhr.open('POST', 'https://<%=Request.ServerVariables("SERVER_NAME") %>/includes/EditorImageUploadHandler.ashx', true);

            xhr.onload = function () {
                if (xhr.status === 200) {
                    var url = xhr.responseText;
                    insertToEditor(url);
                }
            };

            xhr.send(fd);
        }

        function insertToEditor(url) {
            // push image url to rich editor.
            var range = quill.getSelection();
            quill.insertEmbed(range.index, 'image', "https://<%=Request.ServerVariables("SERVER_NAME") %>".concat(url));
        }

        quill.getModule('toolbar').addHandler('image', function () {
            selectLocalImage();
        });


    </script>

Do you see a place where that would belong? I tried it as a separate script and it didn’t work.

@James_Hibbard

quill.clipboard.addMatcher (Node.ELEMENT_NODE, (node, delta) => {
  const plaintext = node.innerText;
  const Delta = Quill.import('delta');
  return new Delta().insert(plaintext);
});

Thanks

[James_Hibbard]

Like this:

// Fetch content from database here
var editorContent = '<%=rs("comments")%>';
var editor = document.getElementById('editor-container');
var hiddenInput = document.getElementById('myHtml');
var form = document.forms.mainform;

var quill = new Quill(editor, {
  modules: {
    table: true,
    syntax: true,
    toolbar: '#toolbar-container'
  },
  placeholder: 'You have lots of space to tell your story!.',
  theme: 'snow'
});

quill.clipboard.addMatcher (Node.ELEMENT_NODE, (node, delta) => {
  var plaintext = node.innerText;
  var Delta = Quill.import('delta');
  return new Delta().insert(plaintext);
});

// Rest of your code

See if that works.

The real question is though, did my demo solve your original problem? If so, then we know that we are on the right track.

[javascript7]

@James_Hibbard

See if that works.

The real question is though, did my demo solve your original problem? If so, then we know that we are on the right track.

You did it again! Placement of that snippet of code was key and obviously worked. So glad it was a relatively small add on to the exiting code. For sure, if it’s not in the correct order, if won’t work. That’s where you vast knowledge of Javascript pays off!

Your demo must have solved the original problem. Also surprising that Quill doesn’t simple add this code to their examples.

Thanks for you help again!

[javascript7]

PS: Edit.

I wanted also to say that it did totally strip out the formatting, including any hyperlinks, etc.

Truly now like Note Pad or NotePad ++

[James_Hibbard]

Oh cool. Glad to hear that you got it working.

[javascript7]

Well, just when you think it’s okay then something else shows up. But I believe it’s the same as the last issue, and maybe the last issue is really this issue.

When I type a bold word and then submit it to the database everything is good, but if you take your cursor at the “o” of Hello and press the return to go to the next line to type or insert a photo, it’s okay, but you then backspace on your browser this ? shows.

When you look at the code in the database you get this:

<p><strong class="ql-size-huge">Hello</strong></p><p><br></p><p><strong class="ql-size-huge"><span class="ql-cursor">?</span></strong></p>

It looks like it’s the span class for ql-cursor.

So I research it to see if that can be replaced with a blank.

entering a newline with bold or italics results in a <span class="ql-cursor></span> with a unicode character inside the span, potentially causing character encoding problems with databases

It looks like this needs to be inserted in the code. Do you agree?

elementValue = elementValue.replace(/**[^<]***</span>/g, "");

Where or how does that get placed in the code? Is it best to stop it from inserting into the database and/or is it look to have it on the post page so it shows as a blank (nothing).

Appreciate your thoughts again.

Thank you

PS EDIT — after several hours of additional research I couldn’t find more on this.

But I had this idea and it worked.

My conclusions is that if I remove and replace:

    <meta charset="utf-8">

To:

<meta content="text/html; charset=windows-1252" http-equiv="Content-Type">

I don’t have any of those issues with the question mark appearing on the page. Several attempts were made and all were good.

The question is, is that a good fix or is that a poor reason for a fix? Appreciate your thoughts. Thank you

[James_Hibbard]

Hi,

@javascript7

It looks like this needs to be inserted in the code. Do you agree?

I cannot reproduce the issue following the steps you outline.
Can you make a minimal demo page where this behavior can be observed and post a link here.

@javascript7

The question is, is that a good fix or is that a poor reason for a fix?

The Windows-1252 character set can display the Latin alphabet, which means that it is probably fine for your use case.

It feels like a bit of a hacky fix to me, but if it works, you need to decide where your time is best spent.

[javascript7]

@James_Hibbard

Can you make a minimal demo page where this behavior can be observed and post a link here

Working on it!

[javascript7]

I am having trouble adding resize handlers to a Quill editor.

Post #14 shows more of the code, but I think this part is in play:

        // Fetch content from database here
        var editorContent = '<%=rs("comments")%>';
        var editor = document.getElementById('editor-container');
        var hiddenInput = document.getElementById('myHtml');
        var form = document.forms.mainform;

        var quill = new Quill(editor, {
            modules: {
                table: true,
                syntax: true,
                toolbar: '#toolbar-container'
            },
            placeholder: 'You have lots of space to tell your story! Click the above link - About HTML Editor for more information.',
            theme: 'snow'
        });

I believe I need to edit the above to include this:

var quill = new Quill('#editor', {
    theme: 'snow',
     modules: {
        imageResize: {
          displaySize: true
        }
    }
});

From there it looks like I need to an more JS, but I am not sure exactly which JS and if more than that is needed. Any of my experimenting is not working.

Here are my sources:

Plunker - Untitled

GitHub - kensnyder/quill-image-resize-module: A module for Quill rich text...

https://codepen.io/knekk/pen/odwELZ

I am sure I am making this more difficult than necessary! Any help would be appreciated.

Thank you