jReviews Support Forum

As I get ready to take a new site live I realize that there will be lots of work to add all of the content items and the custom jReviews fields.  I have my one section and multiple categories set up.  I also have all of my custom jReviews fields setup.  I am now wondering if it is better to pay some to enter all of the data or develop as system to import it.

I am thinking importing may be a good way to go.  I have read about what it takes and wanted to recap it here so others could point out any flaws or possible better ways to do it.

First I think the sections and categories need to be set up.  These are jos_categories and jos_sections.  I already have this done.

Second I think the content items need to be set up.  This table is jos_content.  This would import the descriptions of of the items.  This may difficult to import because you would need all of the raw hthml to get the items to display correctly.

Third Import the jReviews custom field data.  This table is jos_jreviews_content.  This would add all of the custom jReviews fields that would be displayed using the jReviews templates.

Fourth Import the user and editor reviews.  Not sure what tables these are.  I am only interested in importing a default "Not Reviewed Yet" entry for the editor review.

Databases are not my strong suite, I have always been more of a sysadmin or routing guy.  :-)   

Does this seem correct?  Has anyone done this?  Will it be a time saver or will I be fighting an uphill battle all the way?

Does anyone want to earn some cash or know anyone that does?

All comments are appreciated!

jos_jreviews_comments and there's a column there called author, that when set to 1 indicates it's an editor review.

All the steps seem correct. The important thing is that for each new content you add, typically a new content id is generated by incrementing the highest previous id. So before you do this I recommend you export the mentioned tables to csv, open them in excel, add your new rows and reimport them back. That will also give you a better idea of what should go into the tables based on the existing info from a few manual submits.

Hi,
I am looking to import from a csv file. Th problem is as follows: So far I can see +- 5 tables being affected. I only want to import content and not reviews. I also have two directories
Jos_content
Jos_reviews_content
jos_reviews_fireboard
Jos_reviews_fireboard_content
Jos_reviews_catagories

Very confusing - I will have to work out the relationship between all on them.

Anyone had sucess yet?
Regards
Glenn

I have not attempted it yet.

One issue for me is I would still need to gather the content item with the html for import and then import it.

I know that importing will be better for the long run or large setups but I still have to assemble all of the content and information for the custom fields.

I have found it easier to pay some one to add the items directly rather than pay some one to assemble a spreadsheet and then work out how to import the data. Most of my sites contain 30 to 80 review items.

I am sure as I get more review sites under my belt I will improve the process.  Paying people to add the content items has been a huge step in getting sites completed faster.

Hope this helps!

Hi Thanks for the feedback.

My issue is  3600 hotels, 2700 restaurants and 7000 pubs.

I have a spreadsheet of hotels, trying to sort out addresses into correct columes- Ireland is not adress freindly.
i am getting a spreadsheet of restaurants and Pubs done up.

I don't need content yet - just listings.

Thats my story - it will take forever and loads of manhours.

Glenn

Well the number of items you have puts you in a different category  :-)

You have the same issue that I was looking at which is formatting all of the data to get it imported which is a huge task.

When it comes to the actual mechanics of importing the data I would look at Joomlancers for help. http://www.joomlancers.com/ (http://www.joomlancers.com/).  That is where I was going to find expertise.  For me the task of formatting the data was more than just outsourcing the job of adding all of the content items to anyone that wanted to add them at a per submission price.

Good Luck!   

I would love to see the site when you have it done.  I have not had to layout a review site that large yet.  Sounds challenging and fun.  :-)

It's not that complicated.

1) You create your sections and categories. You can use the mass content component which allows you to create many sections/categories very easily. Now you have the category ids you will need to use for each content entry.
2) You create your custom fields
3) You insert some test content and export the data to a csv file from jos_content and jos_jreviews_content. Now you have an example of how the file should look like
4) You search for the highest id in the jos_content table and use that + 1 for the id of your first row. In jos_jreviews_content, the id is the contentid column.
5) You import the two tables separately, jos_content and jos_jreviews_content.

Hi,
Thanks for that!
What about the fireboard plugin and the paid listing plugin?

Regards
Glenn

What about them? The fireboard addon has a tool to automatically create posts from existing content and there's nothing to import for the paid addon as far as know.

Thanks - saw that yesterday. Is there a way to post to the forum under the users who submitted the item and not the one defaulted in the component?


Thanks Glenn

[Admin note: answer is in the Fireboard addon general board]

Hi,
I have imported 3500 items into my database- Yippie.

One issue - In my address fields I set them up as a drop down select - even though they show in the database eg: jr_town = Limerick  they do not show in the frontend unless you physically add them to the field dropdown? is there a shortcut or can i change the type from a dropdown select to a text field via the database, so that it will pick up the data in that field already. I don't want to have to phsically have to add the towns to the select field.

Hope this makes sense.

At least I can import now.

Glenn

Hi,
I changed the database for towns to be text - then it shows in front end -

Problem - My dropdown search does not work - as you now have to type in the town.

So bak to square one.
Can I import all the towns into my select box?
Glenn

Yes. You would need to find the field id in the jreviews_fields table and then use that to import the options to the jreviews_fieldoptions table.

I have had a play around with importing content into jReviews through phpmyadmin, and whilst it is a little learning curve, i do have to admit it really isint to difficuilt to do, so i thought i would give a quick run down on how i did it so that if someone wants to import items they can.

The first step is to export the tables you want to import content into, these are jos_content and there is one named something like jos_jreviews_content depending on what you set your table prefix to at the start, but it is normally jos.

exporting the tables does a few things, but it is only suggested that you export the tables if you have few items on your site, that way the table is small and doesent consume to much space. My suggestion is to export as CSV and then import into excel or openoffice, that way you have a nice list of the items you want to work with. it also gives you an idea of what the fields required are.

a quick tip though, when you export as csv and import into excel, you should set your field-delimiter (the charactor that tells the system its a new field) to something that is not common, i think i used something like | as i had no end of trouble with , as often the data i used to be imported was filled with lots of commas and the system just didn't like it all that much.

but once you have the excel spreadsheet, you will want to see what the highest id is for the jos_content table, your imported items will be that + 1. Now you can include a list of the items you want in your site, basically make the list and copy and paste the title into the title and title alias fields, and then the description or text you wrote into introtext or fulltext, whatever you used really. Once that is done you will want to copy the item id's of the items you added into the jos_content table and copy them into the jreviews_content table. then copy the custom field values you want in there, and check it all, one thing i did notice is that multiple select lists seam to have * as a seperator, in my case this was easy for cusine as they were seperated by a , so i just did find and replace , with *. uploading it was the hard part, firstly there is a size limit to phpmyaadmin so be wary of that one. secondly when you import into the site, it will ask in the csv options, the field titles, be careful with this one, invalid field count generally means there is something not entered, often setting fields with no value to null will fix this one up.

you will need to make sure that you have set the section id's and cat id's in the jos_content table so that the system knows where to put everything, this is relativly simple if you only have one group of items but if you have lots its time consuming as you have to set up each section and catorgy, ideas of fixing this is maybe in your spreadsheet put an extra field just for your own benifit of the grouping, for example, TV's or Stereo,s. you could then copy this into the jos_content table, and filter by each field value, so yo could filter so that each tv record comes up, then copy and paste the applicable field section and cat, then move onto the next value, its a pain but it really is the only way i could see of doing it except for having virtual catogries... do a search for it and see what you think.

i found in my case it was pretty easy once i got the hang of it, but it took awhile, i also had all items in the one section and catogrie so far, and had virtual groups being cusine, i only imported into custom fields (jreviews_content) and joomla content (jos_content) though. i just used the add new posts feature in the fireboard add-on, which timed out a few times but i got there eventually.

if you have a look in each table through phpmyadmin, you will see what fields need values and what don't. that might help in leaving out fields and saving space, but i got about 16,000 items in without missing the limit.

oh also forgot to mention, to import back into phpmyadmin, you will have to save as csv in excel. hope this gives a general idea of how i did it.

as for the custom component, that is something i am getting done at the moment, and from what i have heard from the developer someone else has also had it done as well, where you could have the 1 csv file with all the values you wanted, and essentially map each field to the table it should go into, like title into jos_content, but suburb into jreviews_content. this is good in terms of size as you will get a size limit with phpmyadmin, but shouldn't with this one. once its done i will try and see if i can get it up here for people to have a play with, and make the suggestion to whoever got a component made to do the same if they can. mine will also import xml but it is hard coded for a specific xml style so that part won't be to much use, but hopefully the csv part will.

Hi,
I am the person who had it done - I have the component installed.

I am testing it out with the import of my towns and areas - we will see what hapens.

Regards
Glenn

Hey ;D it works great!
3500 items imported..... with towns and area in dropdowns.

Now for the next directory.

Regards

glenn

just a quick question in regards to the component you had made up, (im having a similar thing just with xml support really is the main difference)

I was wondering what sort of a hosting plan your on, and also the size of the files, csv i assume you are uploading and processing through the component. the reason i ask is im going to do a similar thing but im thinking in my head and worrying because the files are pretty big im wondering how much cpu usage i will use, and trying to work out a way of doing it so that i don't get my account suspended lol.

the other option i was thinking was if it uses alot of cpu then maybe i would run it on my own home server but use the web server (hosting) as the database just while it is being uploaded, much slower but at least limited cpu usage. i know there will be some difference as xml takes abit more usage but just after some experiences with your component and how it is all going for you.

cheers - Trav

Hi,
Mine does csv files only.
The one file was 3mb and the other was 2mb  and third was about half a meg- no problem at all.

I use hostdime.com - I have a vps server and will move over to my own dedicated one soon.
I have been with them over 4 years now, and have 24/7 support.
I am not punting them. I am also not responsible for anyone wishing to move. Please investigate all options. Do you homework. Cheap is not always best.

I have a xml file whick I am going to convert to csv and upload.

regards
Glenn

Thats okay, its good to be able to have a chat to other people who are using jReviews and Joomla, and get a little bit of an idea on how it is going for them in terms of Hosting, even just knowing who people are using and how they are happy about them. There are so many hosts out there it is hard to know which are good or bad.

at the moment i am using Inmotion Hosting, which seams alright, but i have to admit my site is still in its infancy with few visitors and not really much content. Out of intrest how many visitors do you get per day to need vps or was that just a personal choice?

Its amazing how many dedicated hosts there are out there as well, i even saw one cheap as, like 29 a month, however that said they charged that for reboots alone lol.

Whats your experience in hosting, as in do you have some experience in dedicated hosting or are you looking the managed dedicated.

----------------------------------

Also just to preface for you, don't worry about being responsible for anyone wishing to move, i ask questions based on personal experiences other people have had. and each person is responsible for their own actions, its just nice to get other peoples experiences.

This is a terrific thread.  Great work.  I'm working with a test site that we created using SOBI2 business directory and the problem is twofold, first importing is complicated but not insurmountable using php scripting . . . stilll a bit tedious.  Second and most important is that we are limited to jReviewsEverywhere and therefore we seem to be limited with respect to the features that are native in jReviews. 
So, I'm wondering what component are you using to hold your data.  Also, who did forum members use to do import of data. I'd be interested to compare cost of using an outside contractor.
Many thanks for your kind attention to this.
John

i cant really comment on using outside contractors to import data, but in terms of doing it myself, the easiest way seams to be import your data into excel to transform, i.e work out what columns go to where, and then import as csv through phpmyadmin. That really only works well using jReviews as the way the tables are set out is really smart (well done if your watching lol). Its pretty much as easy as giving the items you are importing unique itemid's and then using the same item id's to import it into the jReviews table with the custom fields.

In terms of using jReviewsEverywhere, i am going to be having a similar problem as well, i am looking at putting a few businesses in there as well using sobi2. That said sobi2 is a really difficult component in terms of the way the components are set out. There are a few posts on their site about importing data, ones to look out are a new script coming soon to import a csv file into database:

http://www.sigsiu.net/forum/index.php/topic,7229.0.html

And the HowTo document on how to import data:

http://www.sigsiu.net/forum/index.php/topic,1438.0.html

Hope this helps a little bit!

I am watching, but let me point out that even though *it seems* the jReviews way is the smart way, it's actually only good from a human's ( as apposed to a machine's) perspective because the format is easily readable by people. I am moving towards a similar table design for the fields in a future version that will enable some additional features and large number of fields without unnecessarily making a table bigger and bigger based on the number of custom fields.

Importing then will be more difficult. The good news is I will have to come up with a script to migrate existing data, so I could use that as a starting point to import from CSV.

Guys, checkout the latest announcement:

http://www.reviewsforjoomla.com/option,com_smf/Itemid,34/topic,3459.msg14159/#msg14159

About CSV Import for Joomla and jReviews

CSV Import allows users to easily create Joomla content items from data stored in spreadsheets and CSV files. When used with jReviews, the powerful review and rating extension that also allows for the creation of more complex structured content items, CSV Import allows for the easy transfer of data from spreadsheets into your jReviews-powered product catalogues, directory listings, contact lists, and more. With or without jReviews, CSV Import is an essential time-saving tool for any Joomla site.

travisd: Thanks for the detailed post! :)