SitePoint Sponsor

User Tag List

Results 1 to 18 of 18
  1. #1
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)


    I am a database consultant in the UK and I was hoping to collect opinions (informed or otherwise) about whether an idea I have to make money is clever or daft.

    BTW- you are welcome to steal my idea since I "stole" it from a book anyway!

    What I've done is used a database (FileMaker) to automatically generated ordinary static HTML pages. This is for a client who sells used notebook PC's and who wanted to display a table on his site showing his current stock. What this means is that within the database there is a button which when pressed causes 10 different HTML tables to be created along with a directory containing about 400 simple HTML files. This happens on the local hard disk and the tables give a summary of my client's stock and the HTML files are detailed information on each notebook PC which he has available. These files still need uploading etc- they are just ordinary HTML files that have been automatically generated. Automatically creating these files takes about 90 seconds.

    My idea is to concentrate my consultancy on doing this for others since it seems pretty useful and might be something that lots of people would like to have.

    That said, it is a "cheap and cheerful" solution to the same problem addressed by building a database driven site. What I would love to know is:
    -Are others doing this?
    -Would some of your clients benefit from this type of approach or am I wasting my time thinking about this - should I just learn PHP and MySQL?

    Thanks for your opinions!

    Cheers,

    Chris Van Buren

  2. #2
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think you should just learn how to intergrate a database with a website.

    You already have the information in DB form, and then you turn it into html.

    Why not just keep it in db form?

    No offense but this seems like a backwards step.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  3. #3
    SitePoint Wizard
    Join Date
    Mar 2001
    Posts
    3,537
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hi,

    It sounds like uploading the files to the internet where his clients could see the results would be a hassle, and his site is not dynamic; it is a static representation of his current inventory. If he wanted the users of his website to have a real time view of his inventory, he would have to regenerate those files every time he made a sale. Maybe he doesn't really need real time inventory tracking. php and mysql are for dynamic database driven sites where information needs to be displayed dynamically and quickly.

    And, with a dynamic database driven website, you can update and display your database information in less than 90 seconds, and it happens in real time. If your client had 100 units of each model in inventory and he sold 1 notebook pc on average every month, then your solution is fine. If your client keeps 10 units of each model in stock and sells 100 pc's a day, he is going to be updating the static html files all day long and all through the night, until he earns enough money to pay a php/mysql/guru to end his torment.
    Last edited by 7stud; May 8, 2001 at 14:01.

  4. #4
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by aspen
    I think you should just learn how to intergrate a database with a website.

    You already have the information in DB form, and then you turn it into html.

    Why not just keep it in db form?

    No offense but this seems like a backwards step.
    Thanks for your response.

    Technically, you are right- it is a backward step and there's no doubt that this isn't right for many situations but I wanted to explore if it is right for some. For example, my client likes it because it lets him keep control of his site (he knows enough to deal with HTML etc but not with PHP and SQL).

    I can also imagine others being attracted on a cost/ease of implementation basis.

    For example, if a client had a Excel spreadsheet with something like current jobs on it and wanted to put this on his site in a confidential location. This would let his sales people see when their customers will get sorted etc. With my solution I would i) import his data into my utility ii) sort it etc as he liked iii) produce his HTML files. Setting it up from what I've done so far would take me maybe 3-4 hours. The client would then have a utility which at the press of a button would spit out HTML that he could then post. CuteFTP Pro (and may others) could also post it automatically.

    So, I suppose it boils down to some very newbie questions about PHP/MySQL:
    -How long does it take to get something simple up and working?
    -What is the performance of PHP/MySQL compared with static HTML?

    Thanks for having patience with a newbie :-)

    BTW- one obvious point here is that I think you can just save Excel sheets out as HTML which makes my example a bit daft. A better example is where you want something out from the Excel sheet which is a bit more complex (such as the HTML tables listing notebooks and then linking to HTML files giving more detailed information mentioned in my original post).

  5. #5
    You talkin to me? Anarchos's Avatar
    Join Date
    Oct 2000
    Location
    Austin, TX
    Posts
    1,438
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Something similar to this is done on most larger sites that are dynamic but not constantly updateded. What they'll do is regenerate the static html from the database whenever the data is modified. This means that the database is only queried every update, and not every hit to the page.

  6. #6
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Originally posted by Anarchos
    Something similar to this is done on most larger sites that are dynamic but not constantly updateded. What they'll do is regenerate the static html from the database whenever the data is modified. This means that the database is only queried every update, and not every hit to the page.
    Thanks for this. I suppose you can then imagine a strategy for a large site which would then create another market opportunity. This would be to "split" a larger site into two parts-
    i) one which is is and needs to be truly dynamic and which is developed with PHP/MySQL or similar
    ii) another part where the database element really is an "engine" for creating/maintaining the static pages effectively. This is where I could then come in. This is a bit different from my earlier idea of a utility- it really means being the webmaster through the database.

    I suppose the question is: "Is a split worth it?" I believe that FileMaker is very effectively, quick to use etc but have no real basis for comparison. Any comments would be most welcome.

    BTW- you can learn more about this and how FileMaker does this at

    http://www.zdnet.com/devhead/stories...0746%2C00.html

    This is a bit dated but it does seem that Ziff Davis did (maybe still does) generate their whole site with this technique and FileMaker.

  7. #7
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Everything you want to do is done better with a database.

    You can write scripts that will automate every task you want to do, scripts that will allow the site owner to control every aspect of his site without even knowing HTML.

    And with your setup you're putting alot of work into it. Those hours will add up.

    Large sites do not do things like this either. They basically are just caching their queries. They'll build HTML pages or otherwise cache the information once a day or whatever time period they set. For instance I do this on my site, when a page is accessed it will be cached by the server for 11 hours. But its still automated.

    What you want to do is lessen the automation. What you want to do is NOT what larger sites do - its a backwards step. And eventually it'll cost more in man hours then it would be to pay for someone to make an automated solution for you.

    And you're also relying on your client knowing HTML etc. Most people don't, most people can hardly check their email.

    But there's a big problem with many of the database-to-HTML solutions floating around out there: they expect you to be able to run both the database and the special software on the Web server itself. Unfortunately, most of us keep our sites on an ISP's server, so we don't have permission to do things like that. Even if we did, most ISPs are Unix-based, while most of these database-to-Web tools are Windows or Mac-based.
    That is from that ZDnet article.

    Now you know what that tells me? That tells me that article is geared towards people that use Geocities, AOL, etc to host their webpage. That is not an article geared towards professional web developers because part of being a professional is paying for quality web space.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  8. #8
    ********* wombat firepages's Avatar
    Join Date
    Jul 2000
    Location
    Perth Australia
    Posts
    1,717
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Beware user input !

    I did a website for a small WA airline, who among other things wanted to be able to upload an EXCEL file to the site which a script would then parse and update flight schedules etc, - cool , well until they manage to spell BROOME 3 different ways in the same file !( not easy ???)

    This , due to the nature of the script which will cascade fresh data throughout the site meant that 2 new flight schedules were created to BRROME and BROME ! you could even book flights for them!

    SO whilst, OK I can get around that problem- you are then waiting/looking for the next possible error and its solution.

    Now the site has full user admin via web-based form, which a trained monkey could use - and is the way I would steer any such project.

    PHP/MySQL is real easy to learn, you can do a lot of powerful stuff with a little knowledge.
    I would strongly suggest you take a look at the tutorials here at sitepoint - or from a beginners point of view http://hotwired.lycos.com/webmonkey & http://www.devshed.com who also have some excellent starter tut's.

  9. #9
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Thanks for your thoughtful reply.

    You're right about the Ziff Davis quote since I have MySQL and PHP for free with my ISP. I wonder, however, if this just shows that the Ziff Davis thing dates to 1998 when things may have been different.

    I think the key question is the productivity of SQL vs FileMaker. If the time spend developing a fully dynamic database driven site is equal or only slightly more than a a "partial FileMaker hack" then your approach is superior all the time.

    I am sure you approach is superior for many sites but I am not convinced that it is superior all the time.

    To answer this question we really need somebody who is both a SQL and a FileMaker guru... I'll let you know if I find one. Thanks again for your comments.



    Originally posted by aspen
    Everything you want to do is done better with a database.

    You can write scripts that will automate every task you want to do, scripts that will allow the site owner to control every aspect of his site without even knowing HTML.

    And with your setup you're putting alot of work into it. Those hours will add up.

    Large sites do not do things like this either. They basically are just caching their queries. They'll build HTML pages or otherwise cache the information once a day or whatever time period they set. For instance I do this on my site, when a page is accessed it will be cached by the server for 11 hours. But its still automated.

    What you want to do is lessen the automation. What you want to do is NOT what larger sites do - its a backwards step. And eventually it'll cost more in man hours then it would be to pay for someone to make an automated solution for you.

    And you're also relying on your client knowing HTML etc. Most people don't, most people can hardly check their email.



    That is from that ZDnet article.

    Now you know what that tells me? That tells me that article is geared towards people that use Geocities, AOL, etc to host their webpage. That is not an article geared towards professional web developers because part of being a professional is paying for quality web space.

  10. #10
    ********* Callithumpian silver trophy freakysid's Avatar
    Join Date
    Jun 2000
    Location
    Sydney, Australia
    Posts
    3,798
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Firepages, a good relational data model is designed to minimise that type of data corruption. This is the problem with less robust sytems modelling data. Spreadsheets do not have anywhere near the data integrity constraints required as your example shows. A good solution would take that spreadsheet data and transform it to a relational database so as to avoid those type of questions.

    Back to the original topic - one of the points being missed by some of the posters is *where* the data is comming from. Chris has told us that his client is creating and storing the data in a local database. Second, I can see a role for creating semi-static content, where the html is generated through some automated process every so often. The advantage of caching static html is that it eliminates one source of risk - that the mysql server is running and connections are available at the server (if you are in a shared server environment).

    However, I do agree with aspen that the best deal is to automate things as much as possible. To my mind, the best way of achieving this would be a web page control panel interface on the server, where the client may upload a CSV text files which copy the latest data in the database, then press a magic "UPDATE" button and have the scripts on the server update the static HTML. No need to even have a database on the server as the data used in the static pages could be pulled from the CSV files. This eliminates the need for the client to have a database on the server. However, as aspen says - its all a lot of mucking about for what its worth.

  11. #11
    Serial Publisher silver trophy aspen's Avatar
    Join Date
    Aug 1999
    Location
    East Lansing, MI USA
    Posts
    12,937
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Something I may not have been clear about.


    There are 3 main reasons for using a database model.


    1. You can easily change the data inside a page without changing the page itself.

    2. If you completely redo your design you dont have to change the database at all

    3. Because of 1 and 2 and since data is automatically updated this means your maintenence of the site will be way down.

    This doesn't mean that you need to serve dynamic pages.

    You could write a script and set it up on a cron job to create HTML pages off the database every 2 hours or something.

    But you're still using a database backend in that situation. So 1-3 still apply.

    Your solution is basically the same thing - but instead of a script automatically generating the HTML you're doing it manually, sure you're using a program but that program doesn't run by itself, and then you're uploading it the server as opposed to having this done on the server.

    Thats why its a backwards step. You are still using a database backend for your site. You're just doing all of the work on it instead of making scripts that'll do it for you. Its sorta like building a fire to make some soup instead of popping it in the microwave.

    If you go away for a week on vacation how do things get updated?

    You should automate everything you can.
    Chris Beasley - I publish content and ecommerce sites.
    Featured Article: Free Comprehensive SEO Guide
    My Guide to Building a Successful Website
    My Blog|My Webmaster Forums

  12. #12
    ********* Callithumpian silver trophy freakysid's Avatar
    Join Date
    Jun 2000
    Location
    Sydney, Australia
    Posts
    3,798
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    You could write a script and set it up on a cron job to create HTML pages off the database every 2 hours or something.
    Apsen, I agree with most of your arguements and the logic behind their thinking. Also, I must apologise for my previous post which could have been more lucid - but I was distracted while I wrote that .

    What I was trying to get at with my last post was this. Where is the data? I have a little saying that "the data is in the business domain", because that's where it is being created and captured. It seems that the client Chris is alluding to has a system in their office where the inventory is recorded into a filemaker database. I am also guessing that creating a system which completely automates the transfer of the data between the local database and the web server is out of the question. So the question is this; what is the best way to design a system whereby the data that is sitting in the local filemaker database in the client's office can be used in the client's web site? To my mind this involves two issues.

    Firstly, how do distribute the task of generating the html from the data. The consensus from the preceding discussion apprears to be that this task is best distributed to scripts running at the web server.

    The second question is how to get the data from the local system where it is being stored in a filemaker database into the web-site sytem. That's the issue as I see it. Short of creating a fully automated system which might be too expensive and complicated for such a client, a web user interface might be the go. This would involve the client using the web page/form interface to manually upload CSV file/s which have been exported from the filemaker database and which the server side web application can update its data.

    This second question I find very interesting. Its a question of how to achieve systems integration between the data in a database that is part of a local system and a remote web server on the cheap. I would be interested if others have some thoughts on other solutions.

    I hope that's not taking the conversation off topic - but I kind of think that's what is at the crux of Chris's original post.
    Last edited by freakysid; May 9, 2001 at 06:50.

  13. #13
    SitePoint Enthusiast Setac's Avatar
    Join Date
    Nov 2000
    Location
    San Marcos CA
    Posts
    83
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Your original idea can probably be a very cost effective solution for small businesses. I think your challenge will be getting enough money from that group to make it worthwhile.

    As a packaged product for this group, it may be a product they simply cannot understand or figure out why they would need.

    Both of those problems can be handled by smart, creative marketing.

    With the increase in availability of 24x7 DSL in the states, I have more clients with permanent, static connections to the Internet. This is allowing me to create Filemaker and ODBC access points that hosted web sites can refer to or access. Which can remove the need for them to make the updates as it can be automated.

    Good luck.
    Dynamic HTML - Is that a Frisbee based language...

  14. #14
    ********* wombat firepages's Avatar
    Join Date
    Jul 2000
    Location
    Perth Australia
    Posts
    1,717
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Beware User Input !

    OK we have been there, but the point is - unless you design the system/DB that produces the CSV file in the first place, you have no idea what if any integrity is built into that system...

    Hence my problem above, ie if there is user input at anytime before you get the data there is always a chance that the data is going to be wobbly.

    The web-based interface that I made for that job now allows no user error - all the airports / destinations / variables are called from tables allowing no mistakes - I can not stop someone from entering an incorrect ticket price... but I can stop them from entering incorrect airports / etc & if they miss out anything essential then the record is not created.

    Now if the client want to print out a schedule they just use the website administration section.

    My point I suppose is.. where possible, just get the client to enter the data into the DB through an interface that you control - then when it goes pear-shaped they can rightly blame you , anyway entering the data into one system only to parse/recreate that data in another seems a long and winding road.

    methinks!

  15. #15
    SitePoint Addict
    Join Date
    Nov 2000
    Location
    London, UK
    Posts
    223
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Performance is moot

    Unless you're intending to have 10,000's of hits a day, or your server is a 486 running in your garage, the whole issue of performance is more or less moot.
    Most servers have at least 1ghz (total), and plenty of ram, and high performance disks, etc... The issue of whether it will take too many resources to access a database doesn't really matter (the only problem could be your database connections maxing out..)

    The other question is: is it the best way? There are tons of different ways of doing what you want.

    One of the more simple ones would be to just have your client update his things onto an online system using PHP/mySQL, which would then dynamically serve the requests. That's the obvious solution.

    Another solution, which could prove interesting, is simply to use PHP's filemaker extension to interface directly with the FileMaker database you already have. That way, instead of uploading 450 HTML files, you could just upload one FileMaker database whenever it changed. However, if you are not running a dedicated server that you can manage yourself, you'll have a hard time getting your host to install the filemaker extensions.

    Yet another solution, along the same lines as the last one, is to transfer your data to Access, and use ASP to access your Access files (no pun or wordplay intended). You can use Access through ODBC, and therefore, this would be easy.

    You could also make an online system which would generate html files. It could also, perhaps, have them all include a common header and footer, so that you could just update those to change the design. Maybe also it could have special comment code in them, so that you could write updating code later ... but this seems uselessly complicated.

    Therefore, it seems that though your solution is viable, there are better ones. It seems that if your client wishes to update his database at his desk, he should be able to do that. However, he shouldn't have to upload hundreds of HTML files, and neither should you. Therefore, perhaps the best idea is to have your update button upload the database file to a secure area of your website, which your PHP scripts can then dynamically access.

    Thanks for Reading,

    Danny.

  16. #16
    imagine no limitations exbabylon's Avatar
    Join Date
    Dec 2000
    Location
    Idaho, USA
    Posts
    452
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    PHP/MySQL combinations are sooo fast to throw together that I beleive there is no excuse for a website that has dynamicly changing content (more that once a week) NOT to have a dynamic site!!! I put up LoadSwap.com, my pride and joy in less than 3 weeks... my first PHP/MySQL project (Or any server side anything for that matter).

    Your solution is great if a company is wanting to convert data ONCE.. if it is their first time going online then you will want to be able to convert a LOT of data... sure.... but, why? Isn't it easier to make a PHP script to parse the data file and then insert it into a MySQL database?

    I highly reccommend and prefer PHP/MySQL, yet I can say that the same end can be accomplished with other databases and programming languages!

    Something to think on.... advanced CMS systems are a day's worth of hard work... then all you have to do is change ONE file to change the entire look of your site!

    God Bless,

    Alex Stanton
    Blamestorming: Sitting around in a group discussing why a deadline was missed or a project failed and who was responsible.

    Exbabylon- Professional Internet Services

  17. #17
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Some Clarification

    Thanks to all for the interesting posts- they have helped me clarify my thinking.

    Here's some more interesting fuel to the fire:
    -I have learned a bit more and now believe that I can construct a 100% automated system. There are plugs-ins for FileMaker now which do FTP and which set off scripts at scheduled times. This means that I believe I can imagine a system which logs on at midnight, does its updates and works with no user involvement. My client just leaves his server running at night, uses his database as normal and that's it.
    -Uploads of 500 HTML files shouldn't be needed as I can script on the local database to tell what has been changed. The tables which summarize things will probable need to be updated pretty regularly but the 500 "item" files won't have that much change.


    Originally posted by aspen
    Something I may not have been clear about.


    There are 3 main reasons for using a database model.


    1. You can easily change the data inside a page without changing the page itself.

    2. If you completely redo your design you dont have to change the database at all

    3. Because of 1 and 2 and since data is automatically updated this means your maintenence of the site will be way down.

    This doesn't mean that you need to serve dynamic pages.

    You could write a script and set it up on a cron job to create HTML pages off the database every 2 hours or something.

    But you're still using a database backend in that situation. So 1-3 still apply.

    Your solution is basically the same thing - but instead of a script automatically generating the HTML you're doing it manually, sure you're using a program but that program doesn't run by itself, and then you're uploading it the server as opposed to having this done on the server.

    Thats why its a backwards step. You are still using a database backend for your site. You're just doing all of the work on it instead of making scripts that'll do it for you. Its sorta like building a fire to make some soup instead of popping it in the microwave.

    If you go away for a week on vacation how do things get updated?

    You should automate everything you can.
    Aspen has things right and his approach is technically superior but I wonder if cheap and cheerful isn't best for many sites. Basically, I guess it boils down to this:
    -I can get the three advantages Aspen mentions without building a true dynamic site.
    -I also gain the advantage of using FileMaker instead of MySQL. How large is this? It is large for me since I know FileMaker but I don't know SQL. If there is anyone who knows both? I would love to hear comments from someone who is well informed.
    -I also wonder if I might gain an advantage in that I may be able to "cut you guys out (sorry)." What I mean is this:
    i) I guess that most sites are built by web gurus who are artistic and know dreamweaver and HTML but who are lost when it comes to SQL.
    ii) To do a dynamic site they need to involve somebody with SQL/database knowledge.
    iii) However, to do a little "hack" on their site with my solution all they have to do is pay me a comparatively small sum to sort things out with FileMaker. This solution also then has the benefit that the web guru can probably "feel" his way through to make changes at a later date.

    The argument back is that really that a SQL guy has been cut out but in his place there is a FileMaker guy. That's true. On the other side, with FileMaker I can suck in data from lots of places (spreadsheets, Access database etc), turn it quickly into a HTML file which can be pretty and have links/pictures etc and put it up on their site. The web guru who designed the static site needs some help from me to set it up but he is still in his comfort zone since everything is still static HTML.

    I don't know- this may be logically flawed. It all comes back to how much nicer/easier/more productive FileMaker is when compared to SQL.

    Another interesting point though is that I could build a "product" targeted to a vertical. For example, for antique dealers I could package things up so that things are very close to their needs which then means that, as long as they are happy to use my database, there is almost nothing else to do- just fill in fields for the names of the jpgs, another field for the ftp address/username/password etc. The designer of their original static site just links to the new table listing products and fills in another field which gives an address for the link back.

    Thanks again for everyone's help!

    Cheers,

    Chris Van Buren

  18. #18
    SitePoint Member
    Join Date
    May 2001
    Location
    UK
    Posts
    7
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Freaky

    I think this topic is now, more or less, dead but I learned alot. I think your point is very important:

    Originally posted by freakysid

    This second question I find very interesting. Its a question of how to achieve systems integration between the data in a database that is part of a local system and a remote web server on the cheap. I would be interested if others have some thoughts on other solutions.
    This isn't what I had in mind at the start but it is actually the strongest argument for what I am proposing. There are many ways you can do this:
    -Do a PHP system and parse some kind of csv file.
    -Blow the cash on the hardware etc to truly have only one database and then link them up.
    -Do the work twice and enter changes on a web form.
    -Make the local database generate static HTML, check for what has changed and ftp up the changes.

    All the options can fit in somewhere and the 2nd option, which has to be the best, becomes cheaper all the time. Another interesting point is that the first option means that you are choosing to have a local database and a web site database which means doing the database development work twice + the tricky work of getting things to parse exactly right. This could be tough if you have lots of different sorts of data that needs to get up to the web.

    Thanks for your help.

    Best Regards,

    Chris Van Buren


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •