Multiple select list results in order - cont'd


#21

The only conversion left to do now is with jQuery, and move the changes to the main index.html file too.

Convert jQuery

Now that jQuery doesn't have to serve double-duty by serving both the global code and the RequireJS code, it's now easier to convert it to work with RequireJS.
Fortunately there are good instructions on how to use jQuery with RequireJS.

First I'll remove the existing jQuery code:

TestRunner.html

<!-- 
    <script src="../lib/jquery-3.2.1.js"></script>
    <script src="../lib/escaperegexp.js"></script>
    <script src="../lib/jquery.focusable.js"></script>
    <script src="../lib/jquery.isscrollable.js"></script>
    <script src="../lib/jquery.filtercheckboxesbytext.js"></script>
 -->
 ```

 The howto says we just need to add a path to the config:

 tests/main.js
 ```javascript
    paths: {
        ...
        "jquery": "../lib/jquery-3.2.1"
    },

Does that work? We can try it with resultscontroller and see if the error message goes away.

resultscontroller.js

// define([], function () {
define(["jquery"], function ($) {
    "use strict";
    // var $ = jQuery;

The other code that needs jQuery is scrollmanager

scrollmanager.js

// define(["scroll"], function (scroll) {
define(["jquery", "scroll"], function ($, scroll) {
    "use strict";
    // var $ = jQuery;

The code in checkboxlist and searchresultscan be updated in the same way.

checkboxlist.js

// define(["scrollmanager"], function (scrollManager) {
define(["jquery", "scrollmanager"], function ($, scrollManager) {
        "use strict";

searchresults.js

define(
    // ["checkboxlist", "resultscontroller"],
    ["jquery", "checkboxlist", "resultscontroller"],
    // function (checkboxList, resultsController) {
    function ($, checkboxList, resultsController) {
        "use strict";
        // var $ = jQuery;

and the only jQuery complaints are about jQuery extensions, such as scrollableparent.

Adding jQuery.isscrollable

We can add jQuery.isscrollable to the shim section of the config:

tests/main.js

    shim: {
        ...
        "jquery.isscrollable": ["jquery"]
    }

and add the location to the paths

tests/main.js

    paths: {
        ...
        "jquery.isscrollable": "../lib/jquery.isscrollable"
    },

and add it as a requirement to the scroll code:

scroll.js

// define([], function ($) {
define(["jquery", "jquery.isscrollable"], function ($) {

Adding jQuery.focusable

The next jQuery module that the test complains about is focusable. We can add that one too in the same way as with isscrollable.

tests/main.js

    paths: {
        ...
        "jquery.focusable": "../lib/jquery.focusable"
    },
    shim: {
        ...
        "jquery.focusable": ["jquery"]
    }

and add it to scrollmanager

scrollmanager.js

// define(["jquery", "scroll"], function ($, scroll) {
define(["jquery", "scroll", "jquery.focusable"], function ($, scroll) {

Adding filterCheckboxesByText

The next jQuery module to deal with is filterCheckboxesByText

tests/main.js

    paths: {
        ...
        "jquery.filtercheckboxesbytext": "../lib/jquery.filtercheckboxesbytext"
    },
    shim: {
        ...
        "jquery.filtercheckboxesbytext": ["jquery"]
    }

and add it to scrollmanager

searchresults.js

define(
    // ["jquery", "checkboxlist", "resultscontroller"],
    // ["jquery", "checkboxlist", "resultscontroller", "jquery.filtercheckboxesbytext"],
    function ($, checkboxList, resultsController) {

And lastly, filtercheckboxesbytext needs escaperegexp, so we can add that as a requirement too.

tests/main.js

    shim: {
        ...
        "jquery.filterCheckboxesByText": ["../lib/escaperegexp", "jquery"]
    }

The global variables have all been replaced now with a modular system using RequireJS.

Updating index.html and main.js

The code all has tests for it now, and progress can now occur by updating the main code.

The index.html file can have all of its scripts removed, other than the require.js one.

index.html

<!-- 
<script src="js/lib/jquery-3.2.1.js"></script>
<script src="js/lib/escaperegexp.js"></script>
<script src="js/lib/jquery.focusable.js"></script>
<script src="js/lib/jquery.isscrollable.js"></script>
<script src="js/lib/jquery.filtercheckboxesbytext.js"></script>
 -->
<script type="text/javascript" src="js/lib/require.js" data-main="js/src/main"></script>

The main config can be copied over from the test main, and the Jasmine parts are the only parts that need to be removed.

src/main.js

require.config({
    baseUrl: "js/src/",
    paths: {
        "jquery": "../lib/jquery-3.2.1",
        "jquery.isscrollable": "../lib/jquery.isscrollable",
        "jquery.focusable": "../lib/jquery.focusable",
        "jquery.filterCheckboxesByText": "../lib/jquery.filterCheckboxesByText"
    },
    shim: {
        "jquery.isscrollable": ["jquery"],
        "jquery.focusable": ["jquery"],
        "jquery.filterCheckboxesByText": ["../lib/escaperegexp", "jquery"]
    }
});
require(["searchresults"], function (searchResults) {
    "use strict";
    searchResults.init();
});

Which leaves us with tests for the code in TestRunner.html, and fully working code in index.html

Here is the code as it currently stands at the end of these updates.
search-results-2018-06-17.zip

The only question that remains now @TechnoKid, is now that we have tests to make further development much faster and easier, are there any issues with the index page results list?


#22

I'm noticing just one issue so far. When you type in a search result then click on a category, it doesn't activate that category until you click on it again.

document.activeElement can be used to investigate what's going on there.

  • When you type in the search field, the active element is the search field.
  • When you click on a category, the search field stops being active (you can tell because the blue border goes away), and the active element is the body.
  • When you click on a category again, the active element then becomes one of the checkboxes.

Investigation will now occur for this issue, after which a test will be added, and code can then be updated to fix the issue.

Edit: It's only when you click on the bolded text of a category that this issue occurs.


#23

Because nothing seems to happen when the bold text is first clicked, the checkbox code comes first under suspicion.

        $(container).on("click", ":checkbox", function (evt) {
            return clickHandler(evt);
        });

Good practice is to protect against this from happening again, so a regression test that finds the problem is needed.

Improving the tests first

Another issue that I spot is that the checkbox tests are clicking the checkbox, whereas in actual usage the checkbox isn't visible and /it's the label that gets clicked on instead. I should do something about that first before moving on to other stuff.

checkboxlist.spec.js

        var items;
        var labels;
        ...
            items = categories.querySelectorAll("input");
            labels = categories.querySelectorAll("label");
        ...
        it("selects the clicked category item", function () {
            // inputs[0].click();
            labels[0].click();
            expect(document.activeElement).toBe(items[0]);
            inputs[1].click();
            labels[1].click();
            expect(document.activeElement).toBe(items[1]);
        });
        ...
        it("doesn't uncheck a checked checkbox", function () {
            // inputs[0].click();
            labels[0].click();
            expect(items[0].checked).toBe(true);
            checkboxList.checkFirst();
            expect(items[0].checked).toBe(true);
        });

Now that everything that the test clicks on are labels that match up with the expected usage, we can go back to adding a regression test for the problem at hand.

Delving into the problem

Placing console.log commands in the click events didn't reveal any likely culprits.
Commenting out searchresults code though did results in finding the likely culprit.

When filtercheckboxesbytext is commented out, the filtering and bolding doesn't occur, and the category only takes a single click to select it.

searchresults.js

        return {
            init: function init() {
                // $(categories).filterCheckboxesByText($(searchField));

Is it the filtering, the bolding, or something else in filtercheckboxesbytext? We can only find out by enabling it and exploring further.

searchresults.js

        return {
            init: function init() {
                $(categories).filterCheckboxesByText($(searchField));

Turning off the bold results in the filtering occurring and a single click only needing to be used.

jquery.filtercheckboxesbytext.js

            $("label", container).each(function highlightMatch(ignore, label) {
                // $(label).html(caseInsensitiveBold($(label).text(), search));
            });

Removing the bold fixes the problem. Why does being bold cause the problem?

index.html

    <label for="chkFruits"><b>xFru</b>its</label>

Bold isn't causing the problem, because the categories still filter correctly, and only a single click is required when clicking on the bold part of the label.

So something else is causing the problem. Even changing it to something as benign as a span results in the problem happening, although replacing it with the same text or different text doesn't result in the problem.

Is it possible to filter out the cause of the problem?

jquery.filtercheckboxesbytext.js

                var highlightedText = caseInsensitiveBold($(label).text(), search);
                if ($(label).html() !== highlightedText) {
                    $(label).html(highlightedText);
                } else {
                    console.log(label);
                    $(label).html(highlightedText);
                }

That doesn't fix the problem, but a clue happens to appear in the log.

When the bold text is clicked, the console log updates again. The first click event isn't being captured by the categories click event, but by the filtercheckboxesbytext change event instead.

Is this the culprit? Let's see what happens when we remove the change event.

jquery.filtercheckboxesbytext.js

        // $(searchField).bind("change keyup", function searchChangeHandler(evt) {
        $(searchField).bind("keyup", function searchChangeHandler(evt) {

That fixes the problem. Let's remove the other changes that we've made.

jquery.filtercheckboxesbytext.js

                var highlightedText = caseInsensitiveBold($(label).text(), search);
                // if ($(label).html() !== highlightedText) {
                //     $(label).html(highlightedText);
                // } else {
                //     console.log(label);
                //     $(label).html(highlightedText);
                // }
                $(label).html(highlightedText);

And the problem remains fixed.

Now that we know what seems to be causing the problem, we need to put change back in the event and create a regression test.

Next up is to find a way to simulate the problem as a regression test, so that we have a failing test that we can make pass by fixing the problem. This helps to protect us if the same problem occurs when other changes are made.


#25

To create a test for the problem, we need to be able to experience it from the test.

Test for the problem

We can edit the filtercheckboxesbytext test so that the sandbox is visible:

filtercheckboxesbytext.spec.js

        beforeEach(function () {
            sandpit = sandbox.init({
                ...
                // visible: false
                visible: true
            });

In the list of tests, we can click on Filter checkboxes by text to have only those tests run, which shows the search line and some categories.

We can now ensure that the test is capable of experiencing the problem by adding a character to the search term. Attempting to click on the bold text of a matching category does result in the same problem happening, which is good news. We can now add a test to simulate that behaviour.

While building the test I can use expect statements to check that my assumptions are correct.

filtercheckboxesbytext.spec.js

        it("Takes only one click (instead of two) to select a bold category item", function () {
            var label = $(categories).find("label:first");
            $(categories).filterCheckboxesByText($(input));
            input.value = "One";
            keyUp(input);
            expect($(label).html()).toBe("<b>One</b>");
        });

That expect statement won't remain, but until the test is complete it helps me to know that I'm on the right track.

How to make the problem occur?

I'm finding it extremely difficult to make the problem occur via testing, due in part to what seems to be browser improvements preventing the easy simulation of keyboard entry.

However, I can check if the change event results in any undue behaviour:

filtercheckboxesbytext.spec.js

        it("Doesn't filter when the change event is fired", function () {
            $(categories).filterCheckboxesByText($(input));
            document.querySelector("#sandbox input").focus();
            input.value = "O";
            $(input).trigger("keyup");
            expect($(label).html()).toBe("<b>O</b>ne");
            input.value = "On";
            $(input).trigger("change");
            expect($(label).html()).toBe("<b>O</b>ne");
        });

Now that we have a failing test due to what the change event does, we can turn off the sandbox visibility:

filtercheckboxesbytext.spec.js

        beforeEach(function () {
            sandpit = sandbox.init({
                ...
                // visible: true
                visible: false
            });

and confirm that the test continues to show that the problem exists.

Fix the problem!

We can now fix the problem:

jquery.filtercheckboxesbytext.js

        // $(searchField).on("keyup", function searchChangeHandler(evt) {
        $(searchField).on("change keyup", function searchChangeHandler(evt) {

and confirm that the failing test is now a green passing test.
The code as it currently stands is found at search-results-2018-06-19.zip

The only other thing to do from here is to figure out if there are any usability issues that we haven't tested by removing the change event.

@TechnoKid - please take a look at the index.html page in the most recent .zip file. Are you aware of any reason why the change event was previously used?


#26

Hi Paul.

Thanks again for all your kind input on this and sorry for the extremely late response; have been spending a lot of time on maintenance of my PC. As you know, it wasn't working correctly and I haven't finished yet!

Anyway, regarding your question, I am not sure as my knowledge is pretty basic. What am I looking at? At my end, in the current code, typing in the search box doesn't narrow down the list and clicking on an item in the list doesn't bring up any divs. Is the file just an example or is it supposed to be functioning?

Thanks Paul.


#27

That's odd, for when I extract search-results-2018-06-19.zip to a folder it works in IE, Edge, Chrome, Firefox, and Safari, it all seems to work without a problem.

What is your testing environment, and what does the js/tests/SpecRunner.html file have to say about things?

Meanwhile, I'll run that testrunner on all browsers that I have, to see if I can dig out any further details.


#28

Using the test runner on IE I can see that the ES6 techniques that I was using arne't understood by IE, so I'm removing template strings and am making a few other adjustments to get the tests working in IE too.


#29

For example:

Template strings

IE doesn't understand the template string syntax:

            html: `<p><input type="text" id="mytextbox"></p>
            <div id="categories">
              ...
            </div>`,

And because this can't be polyfilled, all of the template strings must be replaced with quoted strings.

            html: '<p><input type="text" id="mytextbox"></p>' +
            '<div id="categories">' +
              ...
            '</div>',

Object.remove()

The remove() method isn't understood by IE. It's preferable to polyfill when possible instead of changing code, so a polyfill is added:

SpecRunner.html

    <script type="text/javascript" src="../src/polyfill/element.remove.js"></script>

The element.remove.js file comes from the the childnode.remove() polyfill documentation page.

element.remove.js

// from:https://github.com/jserz/js_piece/blob/master/DOM/ChildNode/remove()/remove().md
(function (arr) {
  arr.forEach(function (item) {
    if (item.hasOwnProperty('remove')) {
      return;
    }
    Object.defineProperty(item, 'remove', {
      configurable: true,
      enumerable: true,
      writable: true,
      value: function remove() {
        if (this.parentNode !== null)
          this.parentNode.removeChild(this);
      }
    });
  });
})([Element.prototype, CharacterData.prototype, DocumentType.prototype]);

Scroll differences

And lastly, browsers have slight differences when scrolling. IE is off by one expecting 317 for the scrolling to the end of the list when most other browsers actually get 316, so the test needs to be updated to check that the amount is close enough.

scrollmanager.spec.js

                // expect(categories.scrollTop).toBe(scrollLen);
                expect(scrollLen - categories.scrollTop).toBeLessThan(2);

And the tests now all work in IE along with Chrome, Firefox, and Safari.


All of those above-changes though only occurred to the testing code. The actual search results code hasn't changed at all and won't affect how the search results code works on IE.

The updated code is found in search-results-2018-07-01.zip

It will be valuable to have you run js\tests\TestRunner.html on the browser that you're having trouble with, to help find out more about the issue that you're facing.


#30

Hi Paul.

Thanks again.

I'm using Firefox Quantum 60.0.1 (64-bit) and SpecRunner.html is a blank white page.

Just to make sure I'm not missing anything, I just need to extract all the files and then double-click on the index.html file, right?

Thanks.


#31

Thanks, I'll download the latest Firefox Quantum and see how it goes.

Yes that's right.

I now have Firefox Quantum 61.0 (64-bit) installed, and the code works perfectly on my end.

Can you please open the web console with Ctrl+Shift+K and tell us if any errors show up there?


#32

Hi Paul.

Yes, please see the errors below.

Loading failed for the with source “file:///home/...../js/lib/jquery.filterCheckboxesByText.js”.
index.html:1

Error: Script error for "jquery.filterCheckboxesByText", needed by: searchresults
http://requirejs.org/docs/errors.html#scripterror
require.js:5:1067

makeError
file:///home/...../js/lib/require.js:5:1067

onScriptError
file:///home/...../lib/require.js:5:13218

I will try it on IE when I am next at work and let you know how it goes. Don't go to too much trouble in finding the problem in Firefox as the main browsers I need it to work on are IE and Chrome at present.

Thanks.

Additional Info: This version of Firefox sure has hidden the javascript options well! In about:config, I can confirm that javascript.enabled is set to 'true' and the web console does not have a check mark for the 'Disable JavaScript' option. Also, https://www.whatismybrowser.com/ tells me that Java is not installed, or is disabled (an so is Flash).


#33

The problem is that you are attempting to run it as a local file. Web browsers don’t let files be loaded in a local environment as a security precautiion to prevent bad people from getting files on your computer.

Put them up on a server and everything will work well.

When testing locally, I use a local server called serve


#34

Do you require assistance on setting up a local server, @TechnoKid ?

Another alternative is to upload the files to a remote server, and run things from there.


#35

OK, thanks Paul.

I can try to see how far I can get and get back to you if I get stuck.

In the meantime, just a question; no need for any action at the moment. Is it possible to have my list of items/results in a separate file, such as Excel, so that I can easily add/remove/edit items and their reference numbers?

Thanks.


#36

With a local server you will access the page with localhost:5000 for example.

You can do it with Node+NPM+Serve, or Node+Yarn+Serve. NPM is a Node package manager, and Yarn is a fancier package manager.

You could instead use a more comprehensive web development server, such as webserver from easyphp, or wampserver, but I recommend keeping things relatively simple with Node+NPM.

Super easy, barely an inconvenience.

A server is definitely needed there, either a local server (see above) or a remote server.

There's a lot of information to keep track of for the items/results. A spreadsheet is useful for humans to access, but it's a bloody nightmare for a program to access because we constantly fiddle with the spreadsheet, breaking things for the program.

I recommend using a JSON config file instead, which is easy to edit and update.

[
  {
    "title": "Fruits",
    "items": [
      "Apples",
      "Blackberries",
      "Raspberries"
    ]
  },
  {
    "title": "Vegetables",
    "items": [
      "Beets",
      "Eggplants",
      "Spinach"
    ]
  }
]

I'll follow up with the details but first, can you supply examples of how reference numbers would be used on the page?


#37

While I want to storm ahead with coding up the conversion of JSON data to HTML code, I must keep in mind that tests must first exist to support all of the code that's written. Because the conversion code is easy, that should make the tests easy to do as well.

Before that though, I must get proper instructions for using this code organised. As a result, I've created the following install.txt file with some basic instructions.

INSTALL
=======

Node (which now comes with NPM) is required for the local server.

Node LTS: https://nodejs.org/en/download/


Use 'npm install' from the command prompt to install the needed modules.

C:\Users\Paul\search-results> npm install


USAGE
=====

Use 'npm start' to start the local server:

C:\Users\Paul\search-results> npm start

The local server then loads in your browser at http://localhost:5000


Use 'npm test' to run the tests:

C:\Users\Paul\search-results> npm test


Errors
======

If NPM can't be found, or if when starting the server there is an error of:
"SyntaxError: Unexpected Token", you need to get an up to date version of Node.

Install Node LTS from https://nodejs.org/en/download/

With that detail dealt with, we're in a better position to make progress with loading the categories information from a separate file.

You can download the project files from search-results-2018-07-11.zip

I await further information about those reference numbers.


#38

Thanks Paul.

Sorry for the unusual late responses; I am am still sorting my pc out.

By reference numbers, I meant the IDs for items in the list, so I could easily reassign the IDs of the items in the list if I were to add a new item in the middle of the list (or remove one), to keep the IDs in order, e.g. 001, 002, 003, etc and not have to have IDs like 002a, etc.

When you said 'A server is definitely needed there', does that apply to the Excel idea only or to JSON also?

Thanks.


#39

Yes, it applies to all file access.


#40

OK, thanks Paul.

Believe it or not, the organisation STILL has not put the original single HTML file on their server! I assume in the work environment, placing everything on the organisations server would make everything work.


#41

There is a difference between development and production. For production everything is squished into just the one JavaScript file.