Ajax "next – prev" won't load some times (smells like data is not escaped)

Hello there, in my homepage i have a block with ajax next-prev

[td_block_18 custom_title="whatever" category_ids="21,23,22,37,-42,-39" limit="6" td_ajax_filter_type="td_category_ids_filter" td_ajax_filter_ids="23,22,37" ajax_pagination="load_more"]

there is a constant flow of new posts which are being displayed through that block, and SOMETIMES the ajax will won’t load anything for some time. Then after some time where enough new posts are inserted it works again. The response im getting when is not working is just a json object with an empty “td_data”

No problem on network or system. The only thing that changes is the posts that are going to be displayed. So? It may be the posts contents? Are the title/text or anything else information that goes from the database through your code is not escaped correctly? Maybe some un-escaped quote or bracket or something from the data messes it up? Unfortunately i can’t reproduce this bug at will, but it does smell like something is not escaped.

Hello there, i found what is causing the issue. It has to do with multibyte (UTF-8) characters, preg replace and json_encode.

If a title ends with capital greek letter pi Π (and possibly other multibyte characters) then something happens between your preg replace and the final json_encode. I guess the preg replace returns malformed characters, which in turn will make json_encode to fail to encode the final output, resulting in the ajax load more articles to fail.

the problem in: td_ajax.php near line 115

// remove whitespaces form the ajax HTML
    $search = array(
        '/\>[^\S ]+/s',  // strip whitespaces after tags, except space
        '/[^\S ]+\</s',  // strip whitespaces before tags, except space
        '/(\s)+/s'       // shorten multiple whitespace sequences
    );

    $replace = array(
        '>',
        '<',
        '\\1'
    );

    $buffy = preg_replace($search, $replace, $buffy);

quick fix to get rid of this thing, temporarily: just comment-out the preg_replace line so it becomes:

// $buffy = preg_replace($search, $replace, $buffy);

i had no time to investigate further due to busy schedule, but i suspect that when multibyte characters pass through preg_replace INTERNALS they are treated as non-multibyte characters, in most circumstances each multibyte character is treated as two non-multibyte characters.

so lets say multibyte “Π” character goes into preg_replace and it is becoming two characters: a weird character(1) followed by a space(2). if whatever preg_replace will do, return the characters as-is, then no problem. “Π” was seen as two characters, but its two halves(1)+(2) got returned again together so no problem. The problem is that if that “Π” is at the end of the post’s title, then it is seen as i said as two characters: a weird character(1) followed by a space(2) BUT that (2) space is matched from your pattern and gets removed. So now preg_replace returns HALF the “Π” character (only the weird (1) character since the (2) space got removed). This makes json_encode to fail due to malformed characters and of course the ajax pagination to fail.

Long-term Solutions? maybe something multibyte safe instead of preg_replace like mb-ereg-replace? Or just simply drop that trimming anyway. Why do it in the first place?

p.s. also it would be unpleasant if a multibyte characters gets split into two characters and one of them is a double quote. just saying.

• This reply was modified 10 years by dimitris.
• This reply was modified 10 years by dimitris.
• This reply was modified 10 years by dimitris.
• This reply was modified 10 years by dimitris.