SitePoint Sponsor

User Tag List

Results 1 to 19 of 19
  1. #1
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    parsing a twitter feed....

    I need some things clarified about how to parse a twitter feed:

    am writing a little twitter search app now.... here are my steps:

    1) do search:

    Code:
    $tweetSearch = $TwitterConn->get('search/tweets.json?q='.$searchForThis.'&count=3');  
    	//	(var $searchForThis comes from the request in pretty conventional fashion..)
    get result as JSON:

    Code:
    	$TwitterConn->decode_json = false;


    now this "JSON" is not really a JSON -- it looks like a JSON, but technically it's a string, yes?
    (meaning: I cannot grab it with jQuery getJSON() method, right? for that it has to technically reside in a .json file? not sure about this....)

    so this is exactly the feed I get from the twitter get() method I mentioned above..
    (man, this looks so messy, can't even tell where one record ends and the next one begins (this one is three records, or entries..) but it validates as JSON.....;-)


    2) so in Javascript to "convert" this string to an object (so I can loop through it) I do:

    Code:
     	var searchResults = <?php echo $tweetSearch?>;
    does this make sense?

    (I don't know how to create objects and loop through them in PHP...;-) that would have to be something like a Hash Map in Java, yes?)


    3) then, to loop through it, I use the standard jQuery method to loop thru objects:


    Code:
    $.each(<?php echo $searchResults?>, function(propName,propValue) {....}
    but this line throws an 'undefined'.... on that embedded PHP variable...


    so what am I doing wrong? am I on the right track in general? I find this Twitter API to be quite challenging actually... the Sitepoint article fretburner posted on a previous thread about this they even talk about the "convoluted" Twitter instructions for their API....;-)


    thank you....

  2. #2
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Hey,

    There's no problem using $.getJSON with a PHP script that returns JSON as its output. Here's an example script, which builds upon the stuff we talked about in your previous threads, that you can adapt for your own purposes:

    index.html
    HTML Code:
    <!DOCTYPE html>
    <html>
      <head>
      <meta http-equiv="content-type" content="text/html; charset=utf-8">
      <title></title>
      <script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-1.10.1.min.js"></script>
    </head>
      <body>
        <form method="post" action=''>
          <input type='text' name='terms'>
          <input type='submit' value='Search'>
        </form>
        <script>
          $('form').submit(function(e){
            e.preventDefault();
            var terms = $("input[name='terms']").val();
            $.getJSON('search.php', {terms: terms}, function(response){
              console.log(response.statuses);
            });
          });
        </script>
      </body>
    </html>
    search.php
    PHP Code:
    <?php
    require_once('twitteroauth/twitteroauth.php');

    $terms filter_input(INPUT_GET'terms'FILTER_SANITIZE_ENCODED);

    if (
    $terms)
    {
        
    $twitter = new TwitterOAuth($consumerkey$consumersecret$accesstoken$accesstokensecret);
        
    $twitter->decode_json false;

        
    $tweets $twitter->get("search/tweets.json?q=$terms&result_type=recent&count=10&include_entities=true");
    }
    else
    {
        
    $tweets '{}';
    }

    header('Content-type: application/json');
    echo 
    $tweets;

  3. #3
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    thank you fretburner....

    for some reason I can't get anything to print on the page...

    Code:
     $('form').submit(function(e){
            e.preventDefault();
            var terms = $("input[name='terms']").val();
            $.getJSON('search.php', {terms: terms}, function(response){
              console.log(response.statuses);    // prints
             $('div#console1').append(response.statuses);   //  doesn't print...
             $('div#console1').append(response);   //  doesn't print...
            });
          });
    (console.log() stmt prints fine... but the append() methods are being ignored....)

    none of this, which I put inside getJSON() method, prints either:

    Code:
      $.each(response.statuses, function(propName,propValue) {	
    		  		
    			if (propName === 'text') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    				
    			if (propName === 'created_at') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    			
    			if (propName === 'lang') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    		  });
    and I don't get any JS errors......

    thank you.....

  4. #4
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    if I do the submission the "old-fashioned" way, i.e., w/o AJAX, then the php just prints this:

    { }


    like no data at all is being returned.....

  5. #5
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by maya90 View Post
    none of this, which I put inside getJSON() method, prints either:

    Code:
      $.each(response.statuses, function(propName,propValue) {	
    		  		
    			if (propName === 'text') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    				
    			if (propName === 'created_at') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    			
    			if (propName === 'lang') {
    				$('div#console2').append(propName + ' : ' + propValue + '<br>');
    			}
    		  });
    and I don't get any JS errors......
    The reason this isn't working is that response.statuses is an array of objects, so when you're passing it to $.each(), propName is the array index and propValue is the tweet object.

    Try this:
    HTML Code:
    <form method="post" action=''>
        <input type='text' name='terms'>
        <input type='submit' value='Search'>
    </form>
    <ul id="results"></div>
    <script>
        (function(){
            var terms = $("input[name='terms']").val(),
                results = $("#results");
            $('form').submit(function(e){
                e.preventDefault();
                $.getJSON('search.php', {terms: terms}, function(response){
                    $.each(response.statuses, function(index, tweet) {	
                        results.append('<li>' + tweet.text + '</li>');
                    });
                });
            });
        })();
    </script>

  6. #6
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    thank you fretburner, but am I not looping through these objects here?

    Code:
     $.each(response.statuses, function(propName,propValue) { ...... }
    isn't this the same as your

    Code:
    $.each(response.statuses, function(index, tweet) { .....}
    with diff var names?
    (jQuery knows whether it's an obj or an array, and assigns that first var accordingly, right?? (an index if it's an array, a string if it's an object... or not?? ;-)

    either way it still doesn't work....

    this is my entire code now.....

    Code:
    
    <!DOCTYPE html>
    <html>
    <head>
    <script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-1.10.1.min.js"></script>
    </head>
    <body>
    
    	<form method="post" action="">
    		<input type="text" name="terms">
    		<input type="submit" value="search">
    	</form>
    	
    	<div id="console1"></div>
    
    	<script>		
    		(function(){
    			var terms = $("input[name='terms']").val();
    			$('form').submit(function(e) {
    				e.preventDefault();
    				$.getJSON('search.php', {terms: terms}, function(response) {
    					$.each(response.statuses, function(index, tweet) {	
    		          		     console.log(response.statuses); 
    					     $('#console1').append(tweet.text + '<br>');
    					});
    				});
    			});
    		})();
    	</script>
    
    </body>
    </html>
    nothing prints on the page..... (console.log() stmt prints fine....)

    ?????

    thank you.....

  7. #7
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by maya90 View Post
    thank you fretburner, but am I not looping through these objects here?

    Code:
     $.each(response.statuses, function(propName,propValue) { ...... }
    isn't this the same as your

    Code:
    $.each(response.statuses, function(index, tweet) { .....}
    with diff var names?
    I changed the variable names to make it clearer what I was doing within the loop. Your original code was written to iterate over the key/value pairs of a single object, but as I explained response.statuses is an array of objects, so when you tried to do this:

    Code JavaScript:
    if (propName === 'text') {
        $('div#console2').append(propName + ' : ' + propValue + '<br>');
    }
    you wouldn't get any output, because propName would never contain any of the object keys.. it would contain array keys (i.e. 0, 1, 2 etc.)

    As for why your current code doesn't work, I pasted it into my test file and I was getting an error in the console saying that e was undefined. I changed the version of jQuery to 1.10.2 and that seems to have solved the problem, and now the code outputs the statuses to the page.

  8. #8
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    thank you fretburner.....

    oh brother... well, mine doesn't work with either jQ 1.8 or 1.10... and I get no errors on e object...
    but I see what you mean... I get confused sometimes, when looping thru an array of objects...;-)

    also, I'm curious as to why you wrapped the function in "()" and all that....;-) (that's like plugin code....;-)

    also: weird.... am getting this error in Chrome only.... (1.8 & 1.10....)

    WTF???

    thank you.....

  9. #9
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    so this would be the correct way to loop?

    Code:
    $.getJSON('sp_process.php', {terms: terms}, function(response) {
    	$.each(response.statuses, function(index, tweet) {
    	
    		$.each(tweet,function(propName,propValue) {
    			
    			if (propName === 'text') {
    				console.log(propName + ' : ' + propValue);
    				$('#console1').append(propName + ' : ' + propValue + '<br>');
    			}
    				
    			if (propName === 'created_at') {
    				console.log(propName + ' : ' + propValue);
    				$('#console1').append(propName + ' : ' + propValue + '<br>');
    			}
    
    		});
    	});
    });
    still nothing prints... and I get that weird error in Chrome.... :-(

  10. #10
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by maya90 View Post
    I'm curious as to why you wrapped the function in "()" and all that....;-) (that's like plugin code....;-)
    The idea of wrapping all the code in a function like that is that then everything inside the function executes in a separate scope and variables (in this case, just terms) don't pollute the global scope. It's called an IIFE (immediately-invoked function expression). It's not really needed in this example, but it's a good habit to get into.

    As for the errors, I'm not sure what might be causing them - certainly your latest code example should work OK. I'll have look into it.

  11. #11
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by maya90 View Post
    also: weird.... am getting this error in Chrome only.... (1.8 & 1.10....)
    Have a look at this page: http://stackoverflow.com/questions/1...-404-not-found - it has instructions about how to get rid of the error message.

    Could you put your code online somewhere I could test it? When I reproduce your code on my machine, it works fine.

  12. #12
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    so now I'm getting that 'e' error, even though am using jQ 1.10..

    oh brother.. this is frustrating....

    ok, fretburner... here's all my code....
    http://www.mayacove.com/dev/tw/tw_api.zip

    it just contains four files:
    -- search.html
    -- search_results.php
    -- and the two includes....


    very much appreciate your help.,

    thank you....

  13. #13
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Right, I think I've finally gotten to the bottom of the problem.. you need to make the following changes:

    Edit all of the PHP files and remove any empty lines and space before the opening <?php tag - it should be the very first thing at the top of the file, otherwise the script will throw a 'headers already sent' error.

    Edit your HTML file and move the terms variable declaration:
    Code JavaScript:
    (function(){
        // Move from here..
        $('form').submit(function(e) {
            e.preventDefault();
            var terms = $("input[name='terms']").val();  // ..to here
            // rest of code not shown
    this one is my fault.. although I'm surprised it didn't cause me problems when I was testing on my laptop

  14. #14
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    oh man... finally!!! finally the feed is printing on my page!!
    thank you fretburner....:-)


    only thing is, the key-value pairs don't print in the same order as I place them in my code... my code:

    Code:
    $.each(tweet,function(propName,propValue) {							
    	
    	if (propName === 'text') {
    		$('#console1').append(propName + ' : ' + propValue + '<br>');
    	}
    	
    	
    	if (propName === 'user') {
    		
    		$.each(propValue,function(userPropName,userPropValue) {
    			
    			if (userPropName === 'screen_name') {
    				$('#console1').append('user : ' + userPropValue + '<br>');
    			}
    		});
    	}
    	
    	
    	
    	if (propName === 'created_at') {
    		$('#console1').append(propName + ' : ' + propValue + '<br>');
    		$('#console1').append('<br>');
    	}
    
    });
    what I get:


    created_at : Wed Aug 28 23:35:21 +0000 2013

    text : 'In My Place' ~ Coldplay 
    user : TheNewSeanKeany
    created_at : Wed Aug 28 23:35:19 +0000 2013

    text : 'Coldplay- the scientist' hermosa canción!
    user : CaamilaAlegre
    created_at : Wed Aug 28 23:35:12 +0000 2013

    text : Yellow by Coldplay has some of the prettiest gutiar ever. It's so beautiful, and I just want to sit outside while its raining and play it.
    user : twirlyenough

    PS: there are no urls for individual tweets? I couldn't find tweet urls in this JSON feed....


    thanks again, FB....:~)

  15. #15
    SitePoint Mentor bronze trophy
    fretburner's Avatar
    Join Date
    Apr 2013
    Location
    Brazil
    Posts
    1,256
    Mentioned
    32 Post(s)
    Tagged
    5 Thread(s)
    Quote Originally Posted by maya90 View Post
    only thing is, the key-value pairs don't print in the same order as I place them in my code...
    The order of object properties in JS is not fixed and varies across browsers. If you want to output properties in a certain order, you can change your code (see my example below).

    Quote Originally Posted by maya90 View Post
    PS: there are no urls for individual tweets? I couldn't find tweet urls in this JSON feed....
    You can create the URL for an individual tweet using the returned data:

    Code JavaScript:
    var tweetUrl = 'http://twitter.com/' + tweet.user.screen_name + '/status/' + tweet.id_str;

    Here's the modified JS that outputs the properties in the order you want, and with a link to the original tweet:

    Code JavaScript:
    (function(){
        $('form').submit(function(e) {
            e.preventDefault();
            var terms = $("input[name='terms']").val();
            $.getJSON('search_results.php', {terms: terms}, function(response) {
                $.each(response.statuses, function(index, tweet) {
                    $('#console1').append('text : ' + tweet.text + '<br>' 
                                        + 'user : ' + tweet.user.screen_name + '<br>' 
                                        + 'created_at : ' + tweet.created_at + '<br>'
                                        + '<a href="http://twitter.com/' + tweet.user.screen_name
                                        + '/status/' + tweet.id_str + '">Original tweet</a><br><br>');
                });
            });
        });
    })();

  16. #16
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    awesome... thank you very very much for all your help and your patience, fretburner.. I really appreciate it.....

    this thread has gotten very long... if I have another question I guess I'll start another thread..

    have a great day.... thanks again....

  17. #17
    SitePoint Zealot txt3rob's Avatar
    Join Date
    Jul 2013
    Location
    Liverpool UK
    Posts
    171
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Have to ask got a copy of the full code that's working I wanted to try grab all tweets with in a set time frame and import them to a database but could only get one result every time due to the nutty array

  18. #18
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)
    the "nutty" array... lol.... that "array" you can get as JSON by using this line...

    Code:
     $twitter->decode_json = false;
    right after this line:
    Code:
     $twitter = new TwitterOAuth($consumerkey, $consumersecret, $accesstoken, $accesstokensecret);

    this is what I use to get the tweets:

    Code:
     $tweets = $twitter->get('statuses/user_timeline.json?screen_name=NewYorker&count=1');
    (you don't need an absolute url here, even if you're on a localhost, the twitter API ads the rest...)

    so this returns a JSON with one record, you copy that entire thing and put it in JSON lint and it will format it for you..(JSON Lint is a very useful JSON-validating tool, and it has the added bonus that it formats the JSON neatly, so it won't look like a "nutty array" anymore....;-)
    (what I do is I copy this newly-formatted json and save it as a .json so I have it handy when I'm parsing it...)


    to grab tweets within a set frame I suppose you can use the "created_at" property in that "nutty array"......;~)
    (i.e., the JSON...)




  19. #19
    SitePoint Guru
    Join Date
    Jun 2009
    Posts
    790
    Mentioned
    2 Post(s)
    Tagged
    0 Thread(s)

    PS: I meant "within a set **time frame....."




Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •