| NewsGrabber Thread |
Since my little NewsGrabber script turns out to be much more popular then I expected I’m starting this little thread. This way you could post your comments over here and I’ll be able to post replies and help out people with problems.
Previous version download: v0.5 |
| Profiles For Download newscientist.com | phpnuke.org | slashdot.net | Blizzard.com | NeoWin.Net | Wired.com |
Latest ReadMe.TXT
NewsGrabber 0.6 “Greenfield”
============================>>
by civax / cfxweb.net
18.07.2003 Version 0.2 (First Public Release)
21.07.2003 Version 0.3
23.07.2003 Version 0.4 (never released)
10.08.2003 Version 0.5
17.10.2003 Version 0.55 (never released)
07.02.2004 Version 0.6
****************** Changes Section ******************
v0.3 : Fixed grabNews() - added $profiles_dir to global (won’t work without).
Added $link to the profiles to easily parse urls of news.
v0.5 : Added news selection form.
Using a temporary external text file to store news.
Added function to rewrite news items in profiles.
More tests were done on phpnuke 6.8.
v0.55 : Split the 1 big (unmanageable) file to several small ones.
Cleaned useless Debug output left from 0.5
Newsgrabber ‘user name’ now set at config file, for easy editing.
v0.6 : Major change: Script can now process several profiles in a row.
Added another text file (proflist.txt) for profiles queue.
Added another php class (multiGrab.php) to handle the profiles queue.
Added 3 new variables to the config file.
Script can now plug into different CMSs, not only php-Nuke.
Added option to print/avoid printing the debug information.
Changed (slightly) the look of results, to make it look nicer.
–
Thanks to everyone who sent me feedback.
Early versions testers : Bryan Stedman, Frag, Andrew, Jason Garland.
v0.6 sponsored by Jeff Greenfield (1stapproach.com)
***************** Technical Section *****************
What is it?
———–
NewsGrabber will scan sites according to given profiles, will
extract news from them and add them to your submitted news queue
on phpnuke (or any other CMS). It’s fast, It’s free and it’s
modular - which means you can write your own profiles and scan
whatever sites you want.
This script was tested to run as an admin module on phpnuke 6.5
6.8 and 6.9.
License
——-
First, The whole newsGrabber is copyrighted (c)2002 - 2004 by
me. I offer NewsGrabber under GPL (www.gnu.org/copyleft/gpl.html).
It’s free to download, free to use and I take absolutely no
warranty on it’s use or performance. Go read the GPL license if
you’re really interested in the legal issues. I’m not going to
discuss them here.
Installation
————
1. Copy the files exactly as they are sorted in directories.
2. Go to /Modules/NS-NewsGrabber.
3. Chmod all .txt files to 777 (there should be 2 such files).
Shouldn’t be very hard to understand which file goes where.
Profiles
——–
Each site you want to scan got to have a profile, and the profile
must be located in the /modules/NS-NewsGrabber/profiles directory.
The best thing about NewsGrabber is the flexibility - You can have
it grab news from any site you want if you’re willing to make a
profile for it. Making profiles, however, is a little complicated
at first, until you get the idea. However, if you make a good
profile you can share it with others - and use profiles made by
other people. I’m currently posting profiles free for everybody on
my personal homepage. Just zip them to civax@cfxweb.net and I’ll
post them on http://www.cfxweb.net/civax/code.htm#newsgrabber
I’ve added 3 profiles to this release: phpnuke.org, Slashdot.org
and NewScientist.com. You can also find a very detailed and
explained profile example at the profile’s directory (example.php)
***************** Personal Section *****************
History
——-
NewsGrabber is an old script I wrote back in September 2002 to
help me gather news from various sources I had agreements with,
without the need to visit each site, copy-paste the news and
then rewrite them. Out of my laziness came this handy tool…
Why did I release it now
————————
NewsGrabber 0.2 was released when I noticed XDMP thing got hyped
on phpnuke.org, and turns out they do mostly what my script does,
but they ask monthly fee. I then thought of releasing it for the
sake of the people who couldn’t afford paying (like me). However,
XDMP was terminated due to copyright problems, so now there is
actually no one that supply this kind of service for the phpnuke
community. I hope newsgrabber will be of use to some people.
If you wish to read a little about XDMP:
http://www.cfxweb.net/civax/archives/00000173.htm
To use or not to use
——————–
Should you grab news from other sites or shouldn’t you?
Should your site be just a page with borrowed news from all
around the web or should you have original content?
Should your site look like a thousand other nuke sites using
the same themes, with the same modules, same blocks and now
even the same content?
At the bottom line, it’s up to you.
Boring sites will be boring sites with or without NewsGrabber.
Current Version
—————
I first released v0.2 and v0.3 thinking nobody will actually
use it. I was wrong. I also thought the profiles concept is
going to be very problematic for most people. I was right.
As the primary user of the script I first think of new features
I need myself and implements them. You’re all invited to
suggest features, of course, but the script evolves basically
for my own use.
Of course, as I like to keep a head of the last released version,
my personal newsgrabber now supports keywords to choose topics,
something not yet available in 0.6. (I have it back from 0.5 days
but the feature isn’t mature yet to be released).
Version 0.6 is a serious change since it now allows you to choose
several profiles and process them in a queue (as opposed to the
1 by 1 method used up until v0.5). I had this feature requested
for long and only now managed to sit and do it.
Feel free to suggest more features. Remember - the more feedback
I get, the better next version will be!
|
Contact
——-
If you have any questions or remarks, you can post them on this page:
http://www.cfxweb.net/civax/archives/00000171.htm, or contact me at
civax@cfxweb.net.
You can send me profiles you want to share with the community (or
find profiles others decided to share with you) on my personal site
as well. Email zipped profiles to civax@cfxweb.net. Check for shared
profiles on http://www.cfxweb.net/civax/archives/00000171.htm.
Note: Since publishing 0.5 I get lots of emails asking me to make profiles for this and that sites. I have done so in the begining, creating profiles for people as they asked it, but I cannot sit all day and create profiles. The whole point is that YOU will create your own profiles. I’m willing to help in general instructions and guiding, but I will not do the work for you, sorry. I will, however, keep helping people that request help in the email about installation problems. Thanks.
Common Problems
Problems with output file
Have you uploaded the .txt files?
Are they in the right directory?
Have you chmod them to 777?
You must make sure they are writable. You need to chmod them to 777 (writable, readable and executable to all). You can do that either with your favorite FTP Client (if you need a free good one I recommand ACE FTP Freeware) or in shell/telnet/whatever.
If you keep getting Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt ) then something is wrong with your settings. The file is either doesn’t exist in the right directory, or it isn’t writable for some reason.
I get “Processing x” but thothing else shows!
You have a profile problem. Maybe you haven’t filled $newsstart and $newsend. Maybe you haven’t filled the correct format. In any case, you would want to go to the config.php and change $ng_debug to true, to get progress reports.
Reading http://www.newscientist.com/news/ … Done
Removing non-news header … Done
Removing non-news footer … Done
— Parsed -1 news items.
Things to check:
1. You are using the latest version of newsgrabber.
2. Your profiles match the version of newsgrabber (the profiles for each version have the same name but are not exactly the same). If you use newsgrabber 0.5/0.6 look inside the profile file and make sure the Profile History looks like this:
// Profile History:
// 17.07.2003 - created
// 21.07.2003 - added link to format
// 10.08.2003 - Added reWriteNews() function
If you’re not sure or don’t know how to look inside the file just copy the profiles again from the last archive you downloaded.
3. Check if this only happens to you with one specific profile while the others work fine. In that case, the profile may be damaged (upload it to your site again) or the news site has changed something in his news format, which requires remaking that profile.
Frag and other people asking this :
NewsGrabber is an admin only module. you CAN’T make it run by using http://www.yoursite.com/modules.php?name=NS-NewsGrabber !!!
The script will add a new option to your ADMIN menu and will operate from there.
Civax it work with the slashdotnet profile on a nuke 6.5 installation the other profiles same error whatsoever….
Hmm.. this is wierd. The script only says "— Parsed -1 news items" if it couldn’t parsed ANY news at all. Now, news parsing has nothing to do with the phpnuke version since I wrote newsgrabber first as a stand-alone script therefore it doesn’t use any phpnuke functions. The only function that DOES have any connection with phpnuke is the one that submits the news to the queue and the problem doesn’t seems to be originating from there.
I know of a couple of people that got it running ok on 6.8, I’ll see if they made any changes (I don’t have 6.8 myself).
i think there are problems on 0.3 with author, date, time and links. If i put the author name with the news, it puts the default author on the news of my site.
I’ll release 0.5 this weekend. It is much more advanced and you can also rewrite the news text to some extant.
I am trying to create a profile for http://pio.dalegend.com/index.php and it doesnt wanna work. I made the profile and everything, but when I go to grab the news off of the site, this is what I get:
"Reading …
Warning: implode() [function.implode]: Bad arguments. in /home/s5299839/public_html/vchat20/admin/modules/newsgrabber.php on line 90
Done
Removing non-news footer … Done
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )"
#1. I edited the example.php file
#2. I chmodded the newsdata.txt to 777 as stated in the readme.
I have no clue what to do. can someone fill me in?
I’m having the same problems.. I chmoded newsdata.txt to 777 but still get this message "Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )" It worked once after I chmodded it and then I got the error again.. so I chmodded it again but still continue to get the error.
the file is chmond 777
I modified submitStory(), updated News module and News Admin, now everything is fine.
Thanks, civax.
How about adding a new feature to the admin screen that would allow us build a profile based upon a scanned page. Having a wizard to help us build profiles would rock!!
Hello Im running a gaming clan/League. I can get your newsgrabber to work except for when i create a profile.
I get that implode ordeal which has something to do with grabbing the time etc. By It saying newsgrabber.php Im assauming that its talking about the profile we made, since only news grabber is admin wise correct?
Now I was wondering if You can create a tuturail, or perhaps make a howto, wizard that can walk you thru the ordeal of making compatible working profiles of any site. Im trying to get news from gaming sites etc. Any help will be greatly appreciated….
Ok. After doing some checking the error that comes from the newgrabber.php is this actual error:
$string = implode("\n", file($profile_newspage));
Now That would pertain to this in the profiles:
$profile_newspage = "http://server.counter-strike.net";
Now The problem could be (Im not sure) But It could be that That is not the actual news page?
When i figure it out for sure Ill let everyone know and then ill have some gaming news profiles worked out also..
Still any assistance will be appreciated..
Well Im getting closer n closer, Now i keep getting this:
Reading http://www.blizzard.com … Done
Removing non-news header … Done
Removing non-news footer … Done
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )
hmmmm, its chmod 777, So its Hmmm.
Any Ideas?
Before it was working but it wasnt grabbing the news, Its in this field:
$profile_newsstart = "";
$profile_newsend = "
";
Example of the news scetion:
August 12, 2003
Job Openings: Brand Manager and Assoc. Brand Manager
Blizzard is looking to fill two positions in our marketing department: Brand Manager and Associate Brand Manager. If you’re
interested in helping us take the Blizzard brand to new heights, check out our latest
job posting.
Any idea?
Newsstart and newsend must be fulled. Leaving them blank will cause error.
ColdSteel, here is a base profile for grabbing Blizzard’s news. I only did it for you cause it was 2 min. work.
Wow I was really close to what you have.Oh by the way at the end of the link you have
in your post. I just removed it in the url part and got the file.
Very very close to what ya have. Thank Its helps alot ima try my hand at some and ill let ya know.
sure, email me whatever profile you want to share. I’m happy to works fine for you now!
Hey, anyone have working profiles ? We need to organize all the working ones. I’ll be at the fire station tomorrow, but Sunday I’ll start working on some profiles.
Hope all is ok your way Civax. Oh Rebel, I like iTuned, I grabbed some nice downloads off of ya.
Here are my remarks on the future features suggested by Brett, and might interest some of you.
1. Having a pulldown topics list on the parsed news and then push the news item stright to the database (no need to approve it).
My remark: Well, I’ve thought of that too - and it’s totally possible - but I’ve decided against it for several reasons. First, it’s not only a Topics list. You also have to choose a category. And whether or not you want comments, put it in Home and other things. You may want to edit the text. In addition, the text grabbed is not the exact text submitted. The rewriting function only kicks in and rewrites the text AFTER you chose which news you want to add. The last reason is that, at least for me, I don’t want to have a stream of news from one site and then another stream from another site (3 news from phpnuke, then 5 from nukecops etc). I want to mix them. one news from here the next is from there, this way it doesn’t look like all I do is copy other sites’ news.
So, while it is totally possible to do as you asked, I tend to be against it.
2. "mutate newsgrabber into a free version of XDMP that you manually configure."
My remark: Just for your information XDMP is closed for copyright issues which I’m not fully aware of. I wouldn’t want to be the next target. I can write extended code to grab news automatically on time intervals and grab the new news items, but I believe it will be a mistake.
I strongly object excluding the human factor from the process. NewsGrabber is here to EASE UP things, not to make all the decisions for you. Who runs the site, you or a script? How can the script choose which news is suitable to your site and which isn’t? How can a script decide what topic and category the news item belongs to? How can a script find the little bugs that sometimes occur (wrong link url, broken title, text which is not correct - like when the site you grab news from declares it will be closed for a week).
All in all I don’t think the human factor can be exluded. XDMP was ran by HUMANS, not scripts. The only thing the script did was the automatic polling of news items from XDMP database on a regular time interval. The news were still approved by humans to reach that database.
3. "would be cool if you could setup default topics for articles coming from certain sites"
I guess I can set in the profiles a list of keywords that will set the topic to something specific, but each site got it’s own list of topics and his own profiles that I fear this is an almost an impossible task.
But this is something I’m willing to look into for the next version of newsgrabber.
What I need is a rss reader.
I have about a dozen rss feeds I look at on my site but it would be nice to quick view them and import anything new.
I would still like the ability to make profiles too though for other sites who don’t have rss available.
As far as I remember, phpnuke already supports rss. And with all due respect, RSS are hardly considered news. They are just a title and a link.
If you really insists on fetching rss fedds, find a site which lists those feeds you want and scan it with news grabber, where the profile format is "linktitle" and that’s all.
Strange post there.. don´t know what happened…
Wel, I wanted to add that I´ve done a few more profiles for other sites that works great! This is the only one I haven´t got to work!
Hi Ola,
I keep getting a Parsed -1 on that blabbermouth site as well. I’ll keep working on it this week.
What other sites do you have profiles for ?
I successfully installed Newsgrabber !! It is simply great.
Now I am trying to fix it in order to be able to save the stories not in the nuke_queue table but in another table to be able to process them in a different way than normal PhpNuke stories.
I tried and changed the INSERT query in the newsgrabber.php with the new table name but it appears yet to insert news in the queue table.
Is there any parameter somewhere else to tune ?
Hi Massimo. The only function that deals with inserting the news into the queue is
submitStory. It is a modified version from the Submit_News module. There are 2 places where _queue appears, both on that function. Just change that _queue to whatever you want, but make sure the new table is the same as the queue technically.Hello,
have not been using phpnuke long myself i built a site out of html and as the site grew thought i would be able to manage it but arg too much work ! so your addon works very well and id liek to say thankyou very much fro your hard work and effort,your code worked without any problems and appears most users above seem to be reporting errors so i thought id just say thanks and it works
Hello,
The results of a news scan are correct. However, after I approve submissions the story submission page only has name and title, no story. I am using PHP-Nuke Version: 6.8.Could this have something to do with a limit on the size of a stroy that can be added to nuke_queue table?
Thanks.
Please ignore previous question. I had an error in the $story link that caused the problem.
Thanks.
Hello!
Im trying to make a profile for my own page (www.win-xp.no) and to make it realy easy i made this page http://www.win-xp.no/output.php
View source and u see what I mean by easy
But i cant get it to work!?!?
I’ve made more complicated profiles that worked just great.
Can someone try to make a profile for this page and se if it works?
If u do pls E-Mail me
Sorry for dubble posting
This is what my pofile looks like:
**** cut
This is an awesome module! BTW, here is the profile for ChristianityToday…
< ?php
$profile_name = "christianityToday";
$profile_newspage = "http://www.christianitytoday.com/marriage/index.html";
$profile_format = "linktitletextauthordate";
$profile_newsstart = "";
$profile_newsend = "";
$pre_title = " CLASS=\"feattitle\">";
$post_title = "
";
$adj_title = 0;
$pre_date = "";
$post_date = "
";
$adj_date = 0;
$pre_author = "";
$post_author = "
";
$adj_author = 0;
$pre_text = "";
$post_text = "
";
$adj_text = 0;
$pre_link = " $post_link = " CLASS=\"feattitle\">";
$adj_link = 0;
$pre_var1 = "";
$post_var1 = "";
$adj_var1 = 0;
$pre_var2 = "";
$post_var2 = "";
$adj_var2 = 0;
function reWriteNews($title, $date, $author, $text, $link, $var1, $var2, $site_name, $site_url) (
$link = "http://www.christianitytoday.com/marriage".$link;
$story = "$text
Check it out over here.";
return $story;
)
?>
I guess this script screens certain code, because fields in my previous post were not empty. Sorry about that.
Hello all, I am trying to grab news from a site that posts their news within a java applet…I assume this is probably not possible, but would like someone to confirm this. The site that I am trying to grab from is http://www.thewaterboys.com (a great Irish rock band from the 80’s…) Any help would be greatly appreciated. When I try to grab the news from this site I get the message that I cannot write to my newsdata.txt file (this file is chmoded 777 and I am able to grab from other sites…so I assume it is telling me this because there is nothing there to grab due to the java)
Thanks again,
BoneDaddy
Hello all, I am trying to grab news from a site that posts their news within a java applet…I assume this is probably not possible, but would like someone to confirm this. The site that I am trying to grab from is http://www.thewaterboys.com (a great Irish rock band from the 80’s…) Any help would be greatly appreciated. When I try to grab the news from this site I get the message that I cannot write to my newsdata.txt file (this file is chmoded 777 and I am able to grab from other sites…so I assume it is telling me this because there is nothing there to grab due to the java)
Thanks again,
BoneDaddy
Double posting…nice
sorry for that, not sure how I pulled that off.
BD
I have some of the same problems as others here. I would appreciate any help, I am trying to create a profile for planetside.com and planetsidegaming.com, both are not working. I have went over the instructions several times, worked off the example and then other profiles. I also have problems with some of the posted profiles.
Neowin.net and newscientist have the cannot write to file error.
Slashdot has the parsed 1 news item with nothing showing up.
nuke version is 6.9
am running nuke version is 6.9
followed all the instructions. when click on NS-NewsGrabber link.
All I get is ‘Sorry, such file doesn’t exist…’.
Why is this?
I am also having troubles writing to newsdata.txt file and it is chmod 777.
using phpnuke 6.5
Hi first of awsome job, I would love tu run this script in a couple of my sites.
But I am having typical probs as seen in your post and instructions so let me run this by you and see If you have any ideas.
Settings
PHP-Nuke 6.9 with googletap and other mods
Redhat 9.0 Kernel version 2.4.20-20.9
Apache version 1.3.29 (Unix)
PHP version 4.3.3
MySQL version 4.0.15-standard
Help would be greatly apreciated.
Thanks
Vic
Sorry forgot to post problems:
Reading http://www.phpnuke.org … Done
Removing non-news header … Done
Removing non-news footer … Done
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )
Reading http://www.slashdot.org … Done
Removing non-news header … Done
Removing non-news footer … Done
— Parsed -1 news items.
Now I am running latest release and newsdata.txt is 777
Thanks in advance for the help
Vic
I simply wanted to pull some slashdot news posts down to my site… luckily the script came with a slashdot profile, yay! However, it refuses to write to the newsdata.txt file, even though I have unzipped the script into my ‘base’ phpnuke dir, and all of the files have been sent to their proper locations. Everytime I try to pull down slashdot news, I get : Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )
newsdata.txt is chmod 777, owned by apache / apache
Sorry, forgot to mention :
PHPnuke 6.8
Seems like a great mod. Could be very useful for a site I’m building. Just starting to play around with profile creation. In an effort to save time is there anyone who has a profile for sports related websites such as Sportsline.com or Espn.com or others. If so, I would greatly appreciate if you posted them here.
Regards
Greetings All,
I am using 6.9 and I thought I would try it but…..I don’t think I did anything incorrect including chmod 777 the text file. I get this error….
/modules.php?name=NS-NewsGrabber
Sorry, such file doesn’t exist…
url is http://www.collegesports.com/teams/fsu/index.html
start is
end is
I would like a http://www.cnn.com profile. IF someone could do that for me that would be really cool. Im kinda shocked and reliable news source hasnt been made yet D:
haha thanks!
Good news i found a fix for the no able to write data.txt or whatever.
1. Open index/admin/modules/newsgrabber.php
Search for : "$output_file"
Copy this in its place of what it points too:
"http://yourwebsite.com/nuke/modules/NS-NewsGrabber/newsdata.txt
save upload, and happy news grabbing
Would like philly burbs profile for usa news please
http://www.phillyburbs.com
I think civax forgot about this post
Does anyone have a fix for this Implode issue?
Reading …
Warning: implode(): Bad arguments. in /home/fullacce/public_html/admin/modules/newsgrabber.php on line 90
Done
Removing non-news footer … Done
— Parsed -1 news items.
Found a fix to the implode issue, I changed lines 44 & 45 to reflect
line 44 $profiles_dir = "/home/fullacce/public_html/modules/NS-NewsGrabber/profiles/"; // where are the profiles?
line 45 $output_file = "/home/fullacce/public_html/modules/NS-NewsGrabber/newsdata.txt"; // the name of the text file name to write and now it works, just have to change the SQL format for my newer Nuke verson 7.0
okay…there are alot of posts concerning this issue does anyone have a solution? It seems to be overlooked. I am getting the following error:
Reading http://www.phpnuke.org … Done
Removing non-news header … Done
Removing non-news footer … Done
— Parsed -1 news items.
Here is my enviroment:
Windows 2k
PHPNUKE 6.9
Apache/2.0.46
mod_perl/1.99_10-dev
Perl/v5.8.0
PHP/4.3.2 mod_ssl/2.0.46 OpenSSL/0.9.7b
MySQL 4.0.13
My file structure is as follows:
c:\wampp2\htdocs\bulkerboard\admim\
( I put the files in the correct location in the admin area ie case,links,modules)
c:\wampp2\htdocs\bulkerboard\modules\NS-NewsGrabber\profiles\
c:\wampp2\htdocs\bulkerboard\modules\NS-NewsGrabber\newsdata.txt
(newsdata.txt security settings is set to full control)
newsgrabber.php settings:
$site_name = "Bulkers Board";
$site_url = "http://localhost/bulkerboard/"; // WITH trailing slash!
$profiles_dir = "modules/NS-NewsGrabber/profiles/"; // where are the profiles?
$output_file = "http://localhost/bulkerboard/modules/NS-NewsGrabber/newsdata.txt"; // the name of the text file name to write
I have access to the NewsGrabber from the Administrator consol in PHPNUKE 6.9. When I choose a site to scan I get the before mentioned error.
Someone here has to have a solution!!!!
I’ll admit, I’m fairly frustrated with this very promising module. It doesn’t seem to work for me, and I have a decent amount of experience with PHP-Nuke. Guess I’ll wait for an update or look elsewhere for what I need. Also, considering the author hasn’t replied to a post since September, I’m not too optimistic.
hey guy I have the same problem of the parse.
I think got to do with PHP or at least after mastubating php a couple of hours I managed to make it work. there are 2 variable to set up in php.ini
1) allow_url_fopen = On
2) and enable the string user_agent="PHP"
I think that’s it
Anyway to find out the problem I enabled all the errors on the php-ini and te mistake was at the line 90.
Of course was not the problem of the good script of civax but the configuration of the different programs we use. So After the time spent reading all the article no one was speaking about PHP .. which is the base of phphnuke.
By the way I am not a programmer,
but now is working.
Ciao
Hello I just look around the net for a news grabber and i found a good one, but i have a feedback to tell you and hope you add this feature fore the new version your making.. well here it is
Can you make the script have an option of where the news you grab goes to a ceratain topic ( becouse its kind of messy, all it does it grap news and you need to manualy add it to the topic,
Thanks man
Your a very cool man making this script free for people like us,)
Man this script is good and all, but there is one major problem.
It only read Story Text but not Extended Text so basicly its kind of useless becouse you cant have all the news only..
Hope theres a way to make it work so when it Grap the news from wired.com it will read the Extended Text and put it on the Extended Text box on phpnuke too.
Hi I downloaded the latest newsgrabber, I am running nuke 6.9, all seems to work apart from the cannot write to the newsdata.txt file and it has full access for RW ???? any help would be appreciated
This is a great script. Got it working on nuke 7.0 and found that this script can be used to parse even RSS/RDF feeds. Here’s an example Profile of how I got it working to Parse CNet’s XML Feed http://news.com.com/2547-1_3-0-20.xml .
———————————
// Profile for the site: cnet.com
// XML FEED URL : http://news.com.com/2547-1_3-0-20.xml
$profile_name = "cnet.com";
$profile_newspage = "http://rss.com.com/2547-1_3-0-20.xml";
$profile_format = "titlelinktext";
$profile_newsstart = "";
$profile_newsend = "";
$pre_title = "
$adj_title = 0;
$pre_date = "";
$post_date = "";
$adj_date = 0;
$pre_time = "";
$post_time = "";
$adj_time = 0;
$pre_author = "";
$post_author = "";
$adj_author = 0;
$pre_link = " ";
$post_link = "";
$adj_link = 0;
$pre_text = ""; ";
$post_text = "
$adj_text = 0;
$pre_var1 = "";
$post_var1 = "";
$adj_var1 = 0;
$pre_var2 = "";
$post_var2 = "";
$adj_var2 = 0;
function reWriteNews($title, $date, $time, $author, $text, $link, $var1, $var2, $site_name, $site_url) (
$link = $link;
$story = "$text
Check it out over here.";
return $story;
)
?>
Well it seems to be erroring in the same place as last time, I dont understand why. I would like to get this to work. Here is what happens. (version .6 phpnuke 6.8)
First of all, only 2 of the profiles work directly out of box. This would be the blizzard news, and wired-joystick.
Secondly when I select the news I want and click ok, this happens.
Warning: fopen(/modules/NS-NewsGrabber/newsdata.txt): failed to open stream: No such file or directory in /home/hardtech/public_html/modules/NS-NewsGrabber/processResults.php on line 31
Warning: filesize(): Stat failed for /modules/NS-NewsGrabber/newsdata.txt (errno=2 - No such file or directory) in /home/hardtech/public_html/modules/NS-NewsGrabber/processResults.php on line 32
Warning: fread(): supplied argument is not a valid stream resource in /home/hardtech/public_html/modules/NS-NewsGrabber/processResults.php on line 32
Warning: fclose(): supplied argument is not a valid stream resource in /home/hardtech/public_html/modules/NS-NewsGrabber/processResults.php on line 34
Yes the file is there, and yes the file is CHMOD 777
Im not sure what to do, I am very lost. I would really like to get this working. Ill keep checking around here every chance I get this weekend to see if anyone posted.
Well I found the problem, it was not your scripts fault. But I can into another problem.
Everything checks out, but when I go to the submitted news, it is missing the story. It has the title, but the story is missing.
Do you have any cluse why this may be, and how I might beable to fix it?
all profiles working except slashdot, i’m assuming this has something to do with the start and end portion. someone have a working slashdot profile?
Could someone do a profile for http://www.bluesnews.com ??
also ircspy.com is another good one for gaming news source.
anyone got an idea to why this thing is having problems?!
patrick, download the file again. Seems one of the files was fixed a day after release.
who can solve this problem
——
Processing phpnukeorg
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt
——-
i chamed 777 all the files, i did every thing corectlly but i still have this problem.
i have unix
6.9 nuke.
who can solve this problem
——
Processing phpnukeorg
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt
——-
i chamed 777 all the files, i did every thing corectlly but i still have this problem.
i have unix
6.9 nuke.
Hey. Im having the same problem…
Warning: main(modules/NS-NewsGrabber/config.php): failed to open stream: No such file or directory in /home/mu01/public_html/portal/html/admin/modules/newsgrabber.php on line 62
Fatal error: main(): Failed opening required ‘modules/NS-NewsGrabber/config.php’ (include_path=’.:/usr/lib/php:/usr/local/lib/php’) in /home/mu01/public_html/portal/html/admin/modules/newsgrabber.php on line 62
What can I do to fix this???
Bjorn - seems you have a problem with the path. I had the same error untill I realized my directory name where the module files were was wrong.
It’s working fine now, except the fact I still try to write my own profiles…
how could I make this script run on postnuke?
Well I guess I should update then. It works somewhat yet again. But better
Again only the Blizzard, Wired-Joystick, Wired.com profiles work. The ones that the profile doesnt work I get:
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )
But it clearly can if the threee that work, work semi correctly.
Secondly they have an issue were it shows up as
http://www.yoursite.com/the_correct_link.html
as a link. I would like to know how to get around this and fix this. And if anyone has any idea how to get the rss feeds to work again with this. By chance I took a look at the one posted above, but it doesnt work either.
I would love to see a solution to this, but I do know your busy. But I would rally like ot use this system.
Well i could clarify the yoursite.com thing
basicly wrong site name in front aka yours…
but the rest of the link if it had lets say wire.com in front would work.
So it ends up look like this…
http://www.yoursite.com/news/infostructure/0,1377,62222,00.html?tw=wn_techhead_4
and not what its supposed to be of:
http://www.wired.com/news/infostructure/0,1377,62222,00.html?tw=wn_techhead_4
This seems to be pretty popular in the PHPNuke community. Was wondering if anyone had ever gotten it to work with UBB-Threads. While not classically a CMS there are several modules which give it a CMS look and feel. All I’;ve been able to find for UBB-Threads are news modules which take an RSS feed, but I really need something like this. Anyone know if it’s been done and if so where I may be able to find it?
D-
Did your grabber support not english sites?
Hi All!
I installd it
"Installation
1. Copy the files exactly as they are sorted in directories.
2. Go to /Modules/NS-NewsGrabber.
3. Chmod all .txt files to 777 (there should be 2 such files)."
done.
But i don’t know how can i use this! No index.php or something else…
I activated in the modules in the Admin area, but it alway show me "Sorry, such file doesn’t exist…"
What’s wrong???
Help me!
slashdot profile was not working for me with NG 0.6 and PHPNuke 7.0. I made the following changes to the NEWSSTART and NEWSEND variables after intense trial and error… seems to be working so far… now its time to write some more.
P.S. having problems posting… so i had to omit some lines. sorry.
$profile_name = "Slashdot";
$profile_newspage = "http://www.slashdot.org/";
$profile_format = "titleauthordatetimetext";
$profile_newsstart = "slashdot.org/awards.shtml";
$profile_newsend = "slashdot.org/pollBooth.pl";
$pre_title = "FACE=\"arial,helvetica\" SIZE=\"4\" COLOR=\"#FFFFFF\">";
$post_title = "";
$adj_title = 0;
$pre_date = " ";
$post_date = ", @";
$adj_date = 0;
$pre_time = ", @";
$post_time = "
";
$adj_time = 0;
$pre_author = "Posted by
";
$post_author = "
on";
$adj_author = 0;
$pre_text = "dept.
";
$post_text = "
";
$adj_text = 0;
$pre_link = "";
$post_link = "";
$adj_link = 0;
$pre_var1 = "";
$post_var1 = "";
$adj_var1 = 0;
$pre_var2 = "";
$post_var2 = "";
$adj_var2 = 0;
function reWriteNews($title, $date, $time, $author, $text, $link, $var1, $var2, $site_name, $site_url) (
$story = "$text
Check it out over here.";
return $story;
)
I have written all your comments and will sit to test and - if needed - fix everything the comin weekend. I’m currently hard at work on our game since we have a beta at the publisher in few days.
I’ll post any remarks on this page.
asdf… installation ain’t that hard.
1)upload folder "NS-NewsGrabber" at /html/modules
2)upload the folders case,links , modules at /html/admin folder.
my problem:
i can harverst the news normaly but then i click at the preview/post button at the submision page it directs me at main page
my problem:
i can harverst the news normaly but then i click at the preview/post button at the submision page it directs me at main page
my problem:
i can harverst the news normaly but then i click at the preview/post button at the submision page it directs me at main page
I can’t seem to get Newsgrabber to scan any site, when I try to scan any site I always get:
Warning: grabnews(modules/NS-NewsGrabber/profiles/wired.php): failed to open stream: No such file or directory in /var/www/html/modules/NS-NewsGrabber/parseNews.php on line 31
Fatal error: grabnews(): Failed opening required ‘modules/NS-NewsGrabber/profiles/wired.php’ (include_path=’.:/php/includes:/usr/share/php’) in /var/www/html/modules/NS-NewsGrabber/parseNews.php on line 31
The files are each there and the two text files are chmod 777. i am sorry to bother you with this but have you seen it before or know what could be wrong?
Thanks and sorry to bother you.
I have everything installed like listed but the NS-NewsGrabber Module isnt in my list of available modules?
What am I doing wrong?
Would be nice if you could have target="_new" tags added to the urls so they open in new page. Also, alot of the times it doesn’t grab the url correctly…rather than say http://www.wired.com/blah/blah/news
it will only say /blah/blah/news and forget the wired.com resulting in an invalid page. Other than that its a cool program.
I am having the same problem that Patrick Reynolds stated.
Only Blizzard, Wired-Joystick, and Wired.com profiles work. And with Wired-Technology it apears to get something but shows no listings, it just puts me to the Processing wired-Technology "Add News Queue". On all the others I get "Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )"
All my files are located correctly
1) /html/modules = NS-NewsGrabber
2)/html/admin = case, links, modules
And the 2 txt files have been chmod 777
Of course the other issues with the working profiles is for instance…
tAo8 writes Wired News has posted a new article entitled
1) wired news is a link to the home page of wired news.
2) The link ends up going to http://www.wired.com/news/chode/
3) the fix is to edit out however this would be time consumming as one would ahve to edit all the news posts the make to their site. I have looked for the php file which contains this infor mation but can’t seem to make it work right.
As well "has posted a new article entitled Chode" is printed twice. And "Check it out here" at the bottom is also listed twice.
These plus the fact then when one tries to goto the linked site it take you too the root-relative path instead of the absolute path of the news article. Well I know your busy with the game and all, we used to have dev burns every friday at Black ops. Crap would kill me my friday night at work and then the VP says how about you work this weekend. lol Game Dev ain’t what most peeps invision… Good luck with the game. And thanks for this news grabber and any help.
tAo8
Humm… Should have previewed that post. What it is suposed to say is;
the fix is to edit out the
<
b
/
>
after the link because if you dont it ends up at
http://www.wired.com/news/chode/
<
br
%
20
/
>
k that’s what I ment.
Hi Im running 7.0 php-nuke, will the mod work with the above version? Iv installed and get
Sorry, such file doesn’t exist…
any ideas?
Hey Justin, I a running 7.0 also and it works fine for me except for the prior mentioned bugs. Let us know what the error says explicitly. I mean what file don’t exist. And if it doesn’t say which file then really man it may have issues which are beyond our (general community of users) help; And must be worked out by Da Man who created it. I am gonna take a look at this thing when I get the chance and maybe re-do it. As it seems my man is way too busy with school and all too really pay attention to this little dandy. Right now I have other Phish (heh) in the barrel so I wouldn’t be able to work with it as it really needs some help. I suggest every one look at rdf/rss and php which will give a good idea of how parsing of sites (xml)information is done. Alot can be worked out from that point of view.
My opion is that he never meant this thing to get so big and really don’t wanna spend the time with it or don’t have the time; In which case, I think, some else needs to take it over or at the very least help out with the posts Q&A LoL.
I know how school was (18hrs a day) and I know how it is to have to deal with others who like your S&*% and wanna propagate it for their uses. And I know how the Pro Game Industry is… Because I have worked there, but really don’t let people hang just assign this baby to some one who wants it and let your brain child flow. That’s what Linus did hehe…
Hey tao8
Thanks for the reply, had a look around and ammended the config.php output line as suggested in the phorums. So i think im 1 step closer mmmm….
Im gonna play round till i get it but if you have any ideas let me know.
The error i get in the admin section now..
Reading http://www.phpnuke.org … Done
Removing non-news header … Done
Removing non-news footer … Done
Scan Next Site on Queue ,
Stop Queue, Return to Selection
— Parsed -1 news items.
Regards
I am getting "Sorry, such file doesn’t exist…" when I click on the module off my main menu. I can access the options in the Admin menu fine. Any suggestions?
Wondering how this thing supports postnuke..whle the readme says it supports other cms as well..any ideas. thanks…
Great module. That being said… after many many many hours with the thing. I can say this. It works!
I am using it with Blogger by google so I have complete control over the page that I want to parse. YOu would think it would be a breeze…anyway you must be really exact on the the whole thing. My problem is this. If $story = "$text"; the output to nuke is as it should be the text of the story is returned.
If any character or variable is on the line with $text it will repeat any character around it….so if
$story = "$text $link";
I will get
yadadadadada "the link" "the link".
It’s driving me mad. I worked around it by not using it and using var1 for the text…its working but I must know why! After ohh so many hours of wheel spinning…
Nuke 7.0 on Win2k
Hi, I use Php-Nuke 6.8 on Windows based server.
When I start to scan the default sites, on
http://www.avionet.net/admin.php?op=createList
I receive the following error:
Cannot open file ( modules/NS-NewsGrabber/proflist.txt )
Someone can help me ???
Thanks
hi, i have a problem.
When i post the news to phpnuke, i have in the bottom the "Check it out over here" link, but from some reason it’s write it twice.
Please help me, i can’t fegure it out.
for example:
I got at the bottom of the message
Check it out over here
Check it out over here
I can notget NG to copy to the tx file:
Reading http://www.emol.com/ … Done
Removing non-news header … Done
Removing non-news footer … Done
Cannot write to file ( modules/NS-NewsGrabber/newsdata.txt )
The answer to the problem I could not find anywhere. It looks to me, it might be on the parsing of the page but… I need help. Its a shame people are droping the file due to this problem. Can this be fixed?
Look at UI.php
I suggest you to add
if (!$tRes) (echo ‘empty’; exit) check
in function showAllResults
to resolve "Cannot write to file"
problem.
Look at UI.php
I suggest you to add
if (!$tRes) (echo ‘empty’; exit) check
in function showAllResults
to resolve "Cannot write to file"
problem.
Hi!
I’m trying to make a profile for this site: http://www.dn.se/DNet/jsp/polopoly.jsp?d=1402
All I get in the end is:
Reading http://www.dn.se/DNet/jsp/polopoly.jsp?d=1402 … Done
Removing non-news header … Done
Removing non-news footer … Done
Anyone got any ideas how to make a profile for this rss news site with newsgrabber?
Hi!
I’m trying to make a profile for this site: http://www.dn.se/DNet/jsp/polopoly.jsp?d=1402
All I get in the end is:
Reading http://www.dn.se/DNet/jsp/polopoly.jsp?d=1402 … Done
Removing non-news header … Done
Removing non-news footer … Done
Anyone got any ideas how to make a profile for this rss news site with newsgrabber 0.6?
Does someone work on a postnuke version? If so, could you please tell us when you plan to release it!
Regards
Odika
Hi there
This looks a great script but I think there may be a problem with your download page as I get this error:
"Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@cfxweb.net and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request. "
I hope your able to sort it out soon
Well done with the script
I am getting the same error message when I am trying to download your script and I have tried all sources but the same error happens. I think you may have a problem with the link.
I would love to try this script, can you please email me when the script is available for download.
Thanks.
If the download here does not work then you can still download it from here:
http://www.warp-speed.de/
register, then go to the downloads page, then do a search for "newsgrabber"
you will see it there.
Is there anyone who can help me capture pending legislation from congress.org:
http://www.congress.org/congressorg/issues/votes/
It would be nice to feed in PENDING legislation, so that students can see stuff that’s itnerest them and take action on it BEFORE policy makers vote on it.
Can someone help please?
I am getting the following errors:
Action not permitted
Reading http://www.slashdot.org … Done
Removing non-news header … Done
Removing non-news footer … Done
Scan Next Site on Queue Stop Queue, Return to Selection
— Parsed -1 news items.
…please help.
Warning: filesize(): Stat failed for http://www.distortedcore.com/modules/NS-NewsGrabber/newsdata.txt (errno=2 - No such file or directory) in /home/distor2/public_html/modules/NS-NewsGrabber/processResults.php on line 32
……………help
nevermind im a moron lol
34v3v3 r 3r 3r 3 r3r
Hi guys , hey i have got this script and everything seems to be working well , Would somebody that has a copy of the neowin.net profile be able to send it to me or post it here , the link to it at the top of the page just returns a server error. Any help on this would be wonderfull , You can send it to me at ice2003@tlsforum.com
Thanks
Ice2003
I am not able to install Newsgrabber and each time i click on the icon in admin panel it redirects me to the start of my page. It does not do anything and no errors when i click on the module it says file not found.
Can anyone who has successfully install it post a zip file somewhere so that we can study it.
Regards
Send me a zip file to antispam (at) baijnath.com
I am not able to install Newsgrabber and each time i click on the icon in admin panel it redirects me to the start of my page. It does not do anything and no errors when i click on the module it says file not found.
Can anyone who has successfully install it post a zip file somewhere so that we can study it.
Regards
Send me a zip file to antispam (at) baijnath.com
I am not able to install Newsgrabber and each time i click on the icon in admin panel it redirects me to the start of my page. It does not do anything and no errors when i click on the module it says file not found.
Can anyone who has successfully install it post a zip file somewhere so that we can study it.
Regards
Send me a zip file to antispam (at) baijnath.com
I am not able to install Newsgrabber and each time i click on the icon in admin panel it redirects me to the start of my page. It does not do anything and no errors when i click on the module it says file not found.
Can anyone who has successfully install it post a zip file somewhere so that we can study it.
Regards
Send me a zip file to antispam (at) baijnath.com