Search for REALLY old threads / posts

RudiC · January 12, 2019, 11:26am

Dear friends and colleagues,

for nostalgic reasons I was trying to find an old post. But, unfortunstely, the search for "A Year and older" stopped at 500 entries, somewhere early in 2017.

Do you have any idea on how to get to 2012 / 2013 threads / posts without knowing title nor keywords?

Would it be feasible (in the long run) to allow for the entry of a year to search for?

Thanks and regards
R�diger

Neo · January 12, 2019, 11:34am

Sounds like an easy thing to add to the advanced search function.

Consider it on the TODO list !

bakunin · January 12, 2019, 2:17pm

In the meantime you could use the "sort by" function in the Advanced search and use "Ascending Order" instead of "Descending Order". It might get you the hit within the 500 Posts that show up.

I hope this helps.

bakunin

RudiC · January 12, 2019, 6:15pm

Thank you, bakunin, and shame on me! I was pondering on using that for a split second but decided against it. Now with bakunin rubbing my nose in it, I found: it worked!

Still, all the posts between the first 500 and the last 500 will go unfound, so Neo's announced help is appreciated.

Neo · January 12, 2019, 9:47pm

Hey Rudi,

With :

The badging system first beta basically working and in the testing phase; and
The meta data and HTML5 issues greatly improved resulting in our "unix" keyword search back on page 1 (was #5, #6 when I checked recently depending on various browsers, but is between #6 and #9 in the normal SERP ups-and-downs), and
Countless other forum improvement done using Bootstrap and jQuery, things are going well for the site upgrades.

One of my highest current priorities is to get the "Thank You" button in posts working on mobile. This is a high priority for me. But I am a bit "mentally drained" (or perhaps bored) from all the constant coding, so this upgrade might not be immediate as I have to do some other things in life too (surprise).

However, I agree that adding a "Search By Year" function to the "Advanced Search" form would be useful; but I am sure you can certainly understand the number of users who have so many posts (like you and a few others including me) and a need to search by year for their old posts, is very small compared to the users who could benefit from "Thanks On Mobile" and a few other small upgrade projects; so I have to prioritize. However, I will add that feature you requested in the future, just not now.

Regarding the badging system, I am going to delay calling for comments and let the beta run for a while; and then call for comments later for a number of reasons.

First, I've been coding so much for the badging system that I will not enjoy "Coding For Comments" at this time.
Second, it is best to let the system run for a while and people to get a feel for it; versus just "immediate feelings or reactions" which are not actionable.
Third, as everyone here knows, I prefer solutions rather than comments. If someone comments "I don't like the color scheme of the **** badges" and do not offer an alternative, I am not really not impressed or interested in those kinds of "complaints without a potential solution" comments.

Ravinder says his co-workers love the new system and many have commented "wow" this is one of the best badging systems they have seen, especially on forums. I'm not so excited by either praise or criticism. I just keep working and moving forward, step-by-step.

In closing, this is a technology forum with technical people, so we all should prefer designs and solutions (actionable comments), not opinions without solutions. This is how I was trained by the top brass I used to consult for in the military; and so they always said "bring me solutions, not problems". This is how I think and how I work. I am interested in solutions, not problems. I prefer, or at least hope, even super amazing awk and ksh experts will learn new web development technologies (like Javascript). We should always be learning new things, even when we are expert or very experienced in many things, never stop learning.

It's important to "Keep the Mind of the Beginner" ...which is actually the core of Zen thinking. "Zen Mind, Beginner's Mind"

Cheers and Thanks.

RudiC · January 13, 2019, 7:57am

Hi Neo,

I totally agree that the improved search function is of low priority compared to the current WIP. I'm totally satisfied to find it somewhere on the todo list.
Thanks
R�diger

Neo · January 13, 2019, 9:51am

Cheers.

Its a great idea to add a "search by year" function and so I will do it for sure.

Neo · January 19, 2019, 10:38am

Hey Rudi,

If I post the search.php and the HTML template file are you willing to come up with the PHP and HTML code mods to create your "Search By Year" option for this feature request?

RudiC · January 19, 2019, 11:04am

I am keen to help / contribute, but am afraid my capabilities are limited. I promise to have a deeper look into the project, but I'll have to start from zero with php , code mods, and HTML.

Neo · January 19, 2019, 11:10am

OK.. "mai pen rai" .... I think this is the basic PHP ....

<?php
if ($vbulletin->GPC['searchbyyear'] >= 2000 and $vbulletin->GPC['searchbyyear'] <= $date('Y')) {
    $start_year_unixtime = $mktime(0, 0, 0, 1, 1, $vbulletin->GPC['searchbyyear']);
    $end_year_unixtime = $mktime(23, 59, 59, 12, 31, $vbulletin->GPC['searchbyyear']);
    $thread_query_logic[] = "thread.dateline >= " . $start_year_unixtime . " thread.dateline <= " . $end_year_unixtime;
}

I think I just add this logic above ... and set up the HTML and the HTTPD $_POST for this new $vbulletin->GPC['searchbyyear'] var and we are all set.

Neo · January 19, 2019, 11:14am

WAIT!

That will only give threads you started.....

You need all your individual posts, right?

Or do you need all threads you have posted in?

RudiC · January 19, 2019, 11:29am

We should go for individual posts, as threads would be accessible from there, wouldn't they? And, the limit to 500 posts should be dropped as there could be more in one year.

Neo · January 19, 2019, 11:49am

Yes... I see that

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20180101) and dateline < UNIX_TIMESTAMP(20190101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     1841 |
+----------+
1 row in set (0.03 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20170101) and dateline < UNIX_TIMESTAMP(20180101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     1851 |
+----------+
1 row in set (0.03 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20160101) and dateline < UNIX_TIMESTAMP(20170101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     2502 |
+----------+
1 row in set (0.05 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20150101) and dateline < UNIX_TIMESTAMP(20160101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     2638 |
+----------+
1 row in set (0.06 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20140101) and dateline < UNIX_TIMESTAMP(20150101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     1722 |
+----------+
1 row in set (0.08 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20130101) and dateline < UNIX_TIMESTAMP(20140101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|     2407 |
+----------+
1 row in set (0.15 sec)


mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20120101) and dateline < UNIX_TIMESTAMP(20130101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|      829 |
+----------+
1 row in set (0.12 sec)

mysql> select count(*) from post where dateline > UNIX_TIMESTAMP(20110101) and dateline < UNIX_TIMESTAMP(20120101) AND username = 'RudiC';
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.14 sec)

Neo · January 19, 2019, 11:48pm

Hey Rudi,

Here is the current draft PHP query for this.

Seems to fit the bill what you asked for.

<?php
if ($vbulletin->GPC['searchbyyear'] >= 2000 and $vbulletin->GPC['searchbyyear'] <= $date('Y')) {
    $max_results = 3000;
    $start_year_unixtime = $mktime(0, 0, 0, 1, 1, $vbulletin->GPC['searchbyyear']);
    $end_year_unixtime = $mktime(23, 59, 59, 12, 31, $vbulletin->GPC['searchbyyear']);
    $nl_query_limit = 'LIMIT ' .   $max_results;
    $thread_query_logic[] = "post.dateline >= " . $start_year_unixtime . " thread.dateline <= " . $end_year_unixtime;
}

Neo · January 21, 2019, 5:44am

Hey Rudi,

I think this works as you requested (for Member Name searches only). Basically, I removed the search results LIMIT of 500 for all moderators (this was the easiest fix) and corrected some errors in the PHP (below) image:

Please let me know if you get the results you expect. I checked an it worked fine for me!

<?php
if ($vbulletin->GPC['searchbyyear'] >= 2000 and $vbulletin->GPC['searchbyyear'] <= date('Y')) {
    $start_year_unixtime = mktime(0, 0, 0, 1, 1, $vbulletin->GPC['searchbyyear']);
    $end_year_unixtime = mktime(23, 59, 59, 12, 31, $vbulletin->GPC['searchbyyear']);
    $post_query_logic[100] = "post.dateline >= " . $start_year_unixtime . " AND post.dateline <= " .
    $end_year_unixtime;
}

Admin Notes:

// Does not work as planned so just removed limit for mods in code frag in block below
1401 if(can_moderate() AND $vbulletin->GPC['searchbyyear'] >= 2000 AND $vbulletin->GPC['searchbyyear'] <= date("Y"))
1402 {                                  
1403     $nl_query_limit = 'LIMIT  3000';
1404     $vbulletin->options['maxresults'] = 3000;
1405 }
1406 else
1407 {                                  
1408     $nl_query_limit = 'LIMIT ' . $vbulletin->options['maxresults'];
1409 }
// Does not work as planned so just removed limit for mods in code frag in block below

2655 // trim results down to maximum $vbulletin->options[maxresults]
2656 if ($vbulletin->options['maxresults'] > 0 AND $numitems > $vbulletin->options['maxresults'] AND !can_moderate())
2657 {
2658   $clippedids = array();
2659   for ($i = 0; $i < $vbulletin->options['maxresults']; $i++)
2660   {
2661     $clippedids[] = $orderedids["$i"];
2662   }
2663   $orderedids =& $clippedids;
2664   $numitems = $vbulletin->options['maxresults'];
2665 }

HTML:

search_forums

<div  style="margin:20px 0px 10px 0px;font-weight:bold;cursor:pointer;" ><span>Search by Year:</span><span style="cursor:pointer;margin-left:20px;color:indigo;" class="neo-toolbit" title="User Name Searches by Year in Posts Only. Please Search By User Name and Select Show Results by Posts (Below, Left)"><i class="fas fa-question-circle"></i></span></div>
<div style="padding:$stylevar[formspacer]px">
    <div style="margin:0px 0px 10px 0px;display:none;">:</div>
    <div id="by_year">
        <input type="text" class="bginput form-control" name="searchbyyear" id="by_year_input" size="100"  placeholder="Year in 20YY Format for User Name Searches in Posts"/>
    </div>
</div>

RudiC · January 21, 2019, 8:38am

Looks promising, but unfortuneately:

Sorry - no matches. Please try some different terms below.

If I leave the year open, it yields today's posts (as expected).

EDIT: Hold it, hold it - when I also enter reasonable "Find Posts from" values (e.g. "Yesterday and older"), then it works.

Thanks for this!

Neo · January 21, 2019, 8:55am

Also, Rudi,

According to the logs, you entered an additional field which I did not say to do:

Mon, 21 Jan 19 07:51:02 -0600  UserID: RudiC logic : ["post.dateline >= 1357020000 AND post.dateline <= 1388555999","post.userid IN(302122047)","post.dateline > 1547992262"] 
Mon, 21 Jan 19 07:51:22 -0600  UserID: RudiC logic : ["post.dateline >= 1357020000 AND post.dateline <= 1388555999","post.userid IN(302122047)","post.dateline < 1547992282"]

When I enter the search, this field is not set (from your query)

"post.dateline < 1547992282"

Neo · January 21, 2019, 8:56am

FWIW,

My search produces a different query in the log file:

Mon, 21 Jan 19 07:50:13 -0600  UserID: Neo logic : ["post.dateline >= 1514786400 AND post.dateline <= 1546322399","post.userid IN(302122047)"]

You are doing something in your query different than me, because you have an extra query that I do not have.

RudiC · January 21, 2019, 8:57am

Yes, found it - look at the edit of my earlier post...

Neo · January 21, 2019, 9:00am

Rudi,

Per my instruction, do not enter anything from:

Find Posts from

I specifically stated the ONLY three fields to enter in my post and in the image I provided.

If you don't follow my instructions, it will not work.

That is why I gave instructions. Don't try to break it. I did not ask for a stress test on all possibilities.