SitePoint Sponsor

User Tag List

Results 1 to 22 of 22
  1. #1
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)

    Discourse and other boogey men [bluepill]

    I thought I would do some research. It came about when I started to wonder what bluepill is. As I may have mentioned in another post that I have used rails before but a long time ago. It was version 1.x back then. Much has changed since then ... or has it?

    Back in the version 1.x days I was working on a rails project for a client. We deceloped and tested in Linux and all seemed to go smoothly, well mostly anyway. One day the client suddenly dropped on us that he wanted to deploy to a windows server. Well Apache works on both and so does ruby so it should work, or so we thought, ha ha. What a nightmare! The apache module for rails gave us no end of trouble and was officially unsupported on windows.

    The closest we could get to a solution was to get Apache to reverse proxy to a mongrel server and mongrel was not recommended as a production server in those days. Besides, it felt wrong saying we were running rails in Apache if it was only acting as a proxy.

    We ended up taking a huge risk and rewrote the project in python/Django which was not considered as advanced as rails in those days. Probably due to it being a late starter in that space. However I was pleasently surprised about how much easier it was for me to write in python/Django and it wasn't long before we were comfortable with Django. We then deployed to windows/Apache, using mod_python in those days, and everything just worked. What a champagne moment that was! (Well it would have been if we had any )

    Now of course rails has moved to v3.x with v4 coming soon. Django has gone through several iterations since then also. However I'm still not brave enough to try again and deploy a ruby on rails project on windows

    Anyway, so even though I have used Rails before I had never heard of bluepill. There is so much new stuff around rails that I've scarcely heard of from my v1 days. I became curious about it when I saw it being talked about on meta.discourse.org.

    Well, much to my surprise I found it was for restarting processes that have prematurely terminated. Basically it sits in memory monitoring other processes and if they terminate it restarts them. What processes could this be with Discourse? I am so used to only needing a web server, a database server and a scripting language when writing web applications. I've seldom had to restart those and the only times I've had to is when I've forgotten to run "chkconfig" after installing them. I did have a problem with mysql not starting once but it was due to shutting the machine down suddenly and an orphaned lock file. After deleting the stale lock file I could start it normally.

    So anyway, with Discourse, what is bluepill starting then? Turns out its starting something called thin. Thin, as it turns out, is a web server. The boogey man of reverse proxying returns!!! This is in their instructions for an Apache configuration and now they are running Thin! Huh?? Could this be the reason why Discourse needs or suggests more ram than any other forum in common usage?

    So I say to Sitepoint, when you open this "pandora's box" you are not only getting a rails framework with coding for a forum. You are also getting whatever else is lurking in that box. Bluepill is watching your Thin processes but who is watching Bluepill?

  2. #2
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    I'm really not sure why this is posted here instead of on Discourse's website.... Really it is more of a question for them, one they can give you a better answer on. However, if it puts your mind at ease, Discourse will be hosting the forum. We simply will have a few A records to update so they can use the subdomain we have laid out for it. With that being said, the only thing it will ever see, is Discourse related... so I don't see much of a worry, but then again, I couldn't answer your question about bluepill to begin with (Discourse could though).

  3. #3
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by cpradio View Post
    I'm really not sure why this is posted here instead of on Discourse's website.... [snip]... I couldn't answer your question about bluepill to begin with (Discourse could though).
    Ok, just that I thought Sitepoint staff were rails devs and therefore would make perfect sense to you.

    As to why it wasn't posted on Discourse site was because I wasn't aware bluepill was the sole domain of Discourse development. I assumed it was a normal part of rails development that I had missed out on in my absence from rails.

    Interesting that it's hosted. I wonder what kind of access that gives you to your own plugins and own development.

  4. #4
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by cpradio View Post
    I'm really not sure why this is posted here instead of on Discourse's website....
    I am actually wondering where this statement is coming from. I thought Sitepoint was looking to widening its engagement from just PHP and SEO talk. Ok, maybe I should have posted this to the rails forum but it was hard to know. It was topical and relevant to right now so I posted it here.

    Incidentally I don't ask too much on Discourse Meta as they either tend to ignore me or fob me off. Like the question I asked them for a breakdown of RAM usage was fobbed of with a meaningless response of "awesomeness needs more RAM". I suspect they are following StackOverflow's adage of "suppressing discussion" and only giving short answers.

    I just assumed you guys might be interested in Discourse internals seeing you are looking to use it. Perhaps I guessed wrongly.

  5. #5
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,190
    Mentioned
    191 Post(s)
    Tagged
    2 Thread(s)
    Keep in mind that with the exception of HAWK the rest of the forum Staff are volunteer forum members not SitePoint employees.

    While many of us might be considered programmers, many other's area of interest is in other areas, mark-up, server management, marketing etc.
    Individually none of us is strong in all areas but collectively I think we do well.
    So though some of us can write Ruby, it is the SitePoint employed developers that will be in charge of the back end stuff. Our role is the face of the forums.

    Although the best answers about how Discourse is doing things would come from them, I think the best chance for getting bluepill answers here would be the Ruby & Rails forum, so I'll move it over now.

  6. #6
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    Ok, just that I thought Sitepoint staff were rails devs and therefore would make perfect sense to you.

    As to why it wasn't posted on Discourse site was because I wasn't aware bluepill was the sole domain of Discourse development. I assumed it was a normal part of rails development that I had missed out on in my absence from rails.

    Interesting that it's hosted. I wonder what kind of access that gives you to your own plugins and own development.
    Sorry, maybe I mis-understood your questions or mis-read the verbiage, but I came off of it thinking you were asking how bluepill affected Discourse and any other process thereof on the server (with more emphasis on Discourse). Which is why my recommendation was to ask Discourse (they could tell you better than anyone, even if I studied their source for a few days).

    As for bluepill in general, I'd have to see the Discourse code, but from this article's example, it seems you give it the process name and it watches that process. Now that article was likely a few years ago, as its ruby version is a little far back, so I found these articles too (http://help.cloud66.com/how-to/bluepill.html, and SO http://stackoverflow.com/questions/1...-with-bluepill).

    Since none of those really satisfied my curiosity, I went to the GitHub for bluepill and found in the commands, that it really does work within an application. So with its field of vision limited to the application, I don't really see a lot to worry about (which is visible by looking at the commands, each ends with "for an application").

    Also, I'm not a ruby expert (or rails expert). In fact, I only taught myself some basic Ruby a couple of weeks ago. I'm more of a .NET/PHP guy and I lean harder on .NET than PHP anymore. So all of the above is from my research on Bluepill and not from any experience using it.

  7. #7
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Thanks @Mittineague for moving thread over to correct forum.

    I don't agree that Bluepill does work "in an application" seeing the documentation says it runs as a daemon (or at least as its own process), I know Bluepill itself doesn't use much memory according to the reports.

    What I find disturbing is Jeff Atwood's comments on Meta that implied that Discourse has no bloat. Now I find the standard install is running secondary web servers in their own memory space. Ok, I get running cron jobs to run code that doesn't naturally fit into the http request/response cycle but secondary webservers I go "huh?". I have to wonder what the Discourse dev team are thinking.

    I really am starting to wonder if the Discourse management don't care about how easy or difficult this is to install or whether they rule out shared hosting options like GoDaddy and Hostgator altogether. Their chances of producing a "1 click installer like wordpress" are galloping like wild horses towards the horizon maybe never to be seen again. I guess they are thinking more $$$ for them. Its an ingenious business strategy.

  8. #8
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    I don't agree that Bluepill does work "in an application" seeing the documentation says it runs as a daemon (or at least as its own process), I know Bluepill itself doesn't use much memory according to the reports.
    Read it again... (emphasis mine)
    CLI

    Usage

    bluepill [app_name] command [options]
    For the "load" command, the appname_ is specified in the config file, and must not be provided on the command line.

    For all other commands, the appname_ is optional if there is only one bluepill daemon running. Otherwise, the appname_ must be provided, because the command will fail when there are multiple bluepill daemons running. The example commands below leaves out the appname_.
    It seems to be application controlled. Otherwise, you wouldn't need to provide an app name in a config file or pass it in... Plus since multiple bluepills can run on the same server, it seems even more apparent that it is app driven. Just my take from the documentation.

  9. #9
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by cpradio View Post
    Read it again... (emphasis mine)


    It seems to be application controlled. Otherwise, you wouldn't need to provide an app name in a config file or pass it in... Plus since multiple bluepills can run on the same server, it seems even more apparent that it is app driven. Just my take from the documentation.
    No, I still don't agree. Maybe it can be used that way but that's not the way Discourse is using it. If you look at their installation documentation they are booting up bluepill from rvm. No parameters are passed, they are all read from a configuration file (or a configuration-like file because it looks like ruby script but without the .rb extension).

  10. #10
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    I am referring to the Discourse installation documentation above, not the bluepill documentation (just to try and avoid confusion).

  11. #11
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    I am referring to the Discourse installation documentation above, not the bluepill documentation (just to try and avoid confusion).
    Can you point me to the files in question?

  12. #12
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by cpradio View Post
    Can you point me to the files in question?
    https://github.com/discourse/discour...se.pill.sample

    Also the following under section "Bluepill setup"
    https://github.com/discourse/discour...TALL-ubuntu.md

    It seems from that "bluepill" is being run from "rvm", not executed from a ruby (.rb style) application via any of the usual module imports.

  13. #13
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    https://github.com/discourse/discour...se.pill.sample

    Also the following under section "Bluepill setup"
    https://github.com/discourse/discour...TALL-ubuntu.md

    It seems from that "bluepill" is being run from "rvm", not executed from a ruby (.rb style) application via any of the usual module imports.
    Cool, added to my list to look at today (once I get some caffeine running through my veins). Hopefully, once we are on the same page, I will either 1) see what you are referring to, or 2) spot something that may make more sense (I'd be happy to find either one)

  14. #14
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    https://meta.discourse.org/t/free-hosted-option/9621

    Even though the title of this thread is not really in line with my threads discussion, it does shed light on what those Thin servers are and that they are indeed multiplied for each "configuration instance" and places strong upper limits on the number of Discourse instances a single machine can host. Hence it does in fact seem like Discourse is a "resource hog".

  15. #15
    Programming Team silver trophybronze trophy
    Mittineague's Avatar
    Join Date
    Jul 2005
    Location
    West Springfield, Massachusetts
    Posts
    17,190
    Mentioned
    191 Post(s)
    Tagged
    2 Thread(s)
    Huh? did you post the right link? If so. which post are you referring to?

  16. #16
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Mittineague View Post
    Huh? did you post the right link? If so. which post are you referring to?
    Yep, but it was only post #13 by McKay that was relevant. He explains a bit on how Discourse scales to multiple customers on the same host. (In other words it doesn't particularly.)

  17. #17
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    https://github.com/discourse/discour...se.pill.sample

    Also the following under section "Bluepill setup"
    https://github.com/discourse/discour...TALL-ubuntu.md

    It seems from that "bluepill" is being run from "rvm", not executed from a ruby (.rb style) application via any of the usual module imports.
    I still don't get it, it looks to be the same call that is on bluepill GitHubs repo.
    Code:
    Bluepill.application("discourse", :base_dir => ENV["HOME"] + '/.bluepill') do |app|
    "discourse" is the application name, so it is being provided. It looks like it is using the rvm to read environment data, not start Bluepill. Bluepill has to already be started for you to be able to use it within your application, you are simply telling Bluepill what processes you want watched that are related to your application (sidekiq and thin for example).

    Granted, I still don't know ruby all that well, but that is how it reads to me...

  18. #18
    Hosting Team Leader silver trophybronze trophy
    cpradio's Avatar
    Join Date
    Jun 2002
    Location
    Ohio
    Posts
    5,161
    Mentioned
    152 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Kiwiheretic View Post
    Hence it does in fact seem like Discourse is a "resource hog".
    I won't argue with you on that statement, the high memory/cpu requirements indicated that from the beginning, but Jeff has also openly stated, that their focus is on getting core items developed fully first, then to go back and clean up code to make it perform better on lower requirements (granted, we all know the likelihood of the latter happening is little to none, as what company would want to spend money with no visible improvement/features to show for it?)

  19. #19
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Well, I suspect the founders of StackOverflow woke up one morning and said "Our high page ranking .net platform is looking a bit old and worn. Its become difficult to extend, and it really needs a rewrite. What can we do?"

    "I know", said another, "let's see if we can turn this into a PR exercise. Let's rewrite this platform in RoR, make it open source, be sure to make it bloated enough so it doesn't run on garden variety shared hosting, and start our own hosting offering for $$. People will surely flock to us with their credit cards and we will be seen as the saviours of the internet."

    Quite a brilliant marketing move I think. However I do wonder if Jeff Atwoods is the new Michael "Monty" Widenius (MariaDB) or the old Larry Ellison (Oracle/MySql).

  20. #20
    SitePoint Author silver trophybronze trophy
    wwb_99's Avatar
    Join Date
    May 2003
    Location
    Washington, DC
    Posts
    10,635
    Mentioned
    4 Post(s)
    Tagged
    0 Thread(s)
    ^^^ Not quite. My understanding is Jeff left that bunch and moved on to the next project because StackOverflow was done from a "dream up new venture / app" standpoint. Joel is still flogging SO and didn't come over to discourse.

    As for rails on windows I've done it (IIS AAR + mongrel). App wasn't particularly busy but it stood up pretty well. Biggest issue we had was the "snowflake" deployment, was difficult and tricky to repeat. Reverse proxies in general are a wonderful thing and they make tons of sense for non-php style apps where you've got a persistent app container that serves the app and doesn't handle static files so well as well as all that bad stuff that happens on the internet that leads one to need a hardend IIS / Apache / Nginx setup facing the world.

  21. #21
    padawan silver trophybronze trophy markbrown4's Avatar
    Join Date
    Jul 2006
    Location
    Victoria, Australia
    Posts
    4,108
    Mentioned
    28 Post(s)
    Tagged
    2 Thread(s)
    There seems to be a lot of speculation in this thread about software you don't understand.
    It's reminiscent of the "Rails can't scale" trolling that has been refuted time and again.

    One day the client suddenly dropped on us that he wanted to deploy to a windows server.
    How odd! I haven't had any experience developing or deploying rails on Windows and don't think I ever will.
    I will share a secret with you, people use Macs to make web sites, Linux to run them and Windows to test them in Internet Explorer. Now that's trolling.
    Seriously though, I don't know anyone who would choose to deploy Rails apps to a Windows server. Just don't.

    I've never had to configure servers other than a dev machine myself so I can't comment on the specifics, all I know is that popular Rails architectures can scale perfectly fine for the vast majority of websites. If you get to the size of a Facebook or a Twitter you're always going to need something special.

    More generally, it seems we've been moving toward a more diverse server architecture than in the days of the xAMP stack.
    It's common for web servers to be running many processes like Ruby, Python, a Javascript runtime, Memcache/Redis, Solr/Sphinx etc..

  22. #22
    SitePoint Addict
    Join Date
    Aug 2013
    Location
    New Zealand
    Posts
    277
    Mentioned
    5 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by markbrown4 View Post
    There seems to be a lot of speculation in this thread about software you don't understand.
    It's reminiscent of the "Rails can't scale" trolling that has been refuted time and again.
    What a strange response. Are you saying you are an authority on Discourse? Then you accuse this thread of trolling but you haven't posted any evidence to the contrary. We can all have an opinion but its not very persuasive without evidence. Please explain and enlighten us and include your supporting links.

    To clarify I definitely don't prefer windows as a deployment option, for either Django or Rails, but in the case I had several years ago it was because a client requested it. Rails wouldn't reliably deploy on windows (mainly due to OS incompatible ruby gems) but with Django it was far more successful and painless. However Discourse is written in Rails so thats not the issue here. I never had any desire to deploy Discourse on windows, but I did try to deploy it to Centos which is linux but not officially supported. Interesting to see the Ubuntu partner logo on the Discourse website. Maybe that is the reason why deploying to Centos is so "uninteresting" to Discourse devs.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •