Chad Perrin: SOB

6 September 2009

Where is Scheme for the Web?

Filed under: Geek — apotheon @ 09:54

In Chipping the web: August 27th, smart guy Sterling (aka "Chip") linked to Lisp for the Web.

Lisp for the Web is a familiarization tutorial for doing Web development with Common Lisp, written by Adam Peterson. It uses some portable, open source Lisp libraries to develop a simple but useful Web application (or, well, it would be useful if the subject matter were less silly — but that's easily fixed and doesn't require any coding expertise itself). Adam says, in a bit of foreshadowing about what he'll do with the Web application developed in the course of the tutorial:

  • utilize powerful open source libraries for expressing dynamic HTML and JavaScript in Lisp,

  • develop a small, embedded domain specific language tailored for my application,

  • extend the typical development cycle by modifying code in a running system and execute code during compilation,

  • and finally migrate from data structures in memory to persistent objects using a third party database.

I'll do this in a live system transparent to the users of the application. Because Lisp is so high-level, I'll be able to achieve everything in just around 70 lines of code.

He cheats a little, here — because it would surely require more than 70 lines of code if he wasn't using libraries to do some of the heavy lifting — but that's still probably pretty impressive to the average Java developer who has never used a high-level, dynamic language.

I haven't read the whole thing, but what I have read is very clear, straightforward, and well-presented. In short, it looks great. In fact, it looks like exactly what I would like to have, if it used Scheme instead of Common Lisp.

See, the problem is that I'm working on learning Scheme right now — and using it, in turn, to (re)learn some Computer Science concepts. Part of that involves trying to work my way through the exercises in SICP. Anyway, the point is that I'd like to learn to do this sort of thing in Scheme instead of Lisp, to help with learning practical application of Scheme rather than just getting lost in the land of theory.

There are about three or four such Web development tutorials I've found for Scheme. Unfortunately, they all use a Web development framework that only works on PLT Scheme. Well, hell — I don't want to have to change Scheme implementations every time I want to work on something in a new problem domain. Isn't there something like this that uses portable Scheme libraries, rather than using something particular to a specific implementation?

I know why this kind of problem surfaces, of course. It's because, in every Scheme version up through R5RS, there has never been a standardized module system for libraries. Every implementation of any RnRS such that (<= n 5) uses its own idiosyncratic (and incompatible with others) module system, except in some cases of forks. R6RS specifies a standardized module system, but R6RS is relatively new and somewhat controversial, so it doesn't have nearly the support amongst the old-school Scheme hackers that R5RS and previous versions did. Maybe R6RS will start accreting libraries of the sort that used to be implementation-specific in previous versions, given time, but for now I'm SOL, as far as I've been able to determine.

Bah, humbug. If you know of any exceptions to the "No Scheme Web Development Tutorials Use Implementation Nonspecific Tools" rule, please let me know. Otherwise, it'll probably be a year at least before I get to learn to do Web development in Scheme, since I'll have to figure it out from scratch (well, to some extent, borrow ideas from others' code, but I have to learn to grok others' code well enough to do so) if I don't want to have yet another implementation of the same language installed on my damned laptop. I already have three of 'em (that I remember off the top of my head).

14 Comments

  1. Thanks for the nod.

    Wish I could help you on the Scheme libraries. Of course, the Lisp libraries are all written in Lisp and available as source, so it might be possible to translate them into Scheme. Just a thought...

    Comment by Chip Camden — 8 September 2009 @ 02:48

  2. Yeah . . . if only I knew enough Common Lisp.

    Comment by apotheon — 8 September 2009 @ 09:25

  3. Is PLT Scheme's web framework not fitting your requirements? http://docs.plt-scheme.org/web-server/index.html

    Comment by aaron — 9 September 2009 @ 08:44

  4. As far as I'm aware, PLT Scheme's Web framework only works with PLT Scheme.

    Comment by apotheon — 9 September 2009 @ 10:43

  5. Yes it is right that PLT scheme required for PLT Scheme's Web framework.

    Comment by Popcorn Shirts — 10 September 2009 @ 12:18

  6. It's not going to serve my needs, then. As I said in the above SOB entry:

    There are about three or four such Web development tutorials I've found for Scheme. Unfortunately, they all use a Web development framework that only works on PLT Scheme. Well, hell — I don't want to have to change Scheme implementations every time I want to work on something in a new problem domain. Isn't there something like this that uses portable Scheme libraries, rather than using something particular to a specific implementation?

    What am I supposed to do — change what Scheme implementation I use every single time I want to write a new program, because the libraries I want are only available for a different implementation than whichever I'm currently using? Then, of course, once I write a piece of software, nobody else could use it unless that person installs a new Scheme implementation as well, unless he or she just happens to be using the same implementation.

    This is one of the reasons I selected an R6RS implementation: there's a standardized module system for R6RS. I'm just asking whether there's a Web development tutorial using a portable library — so no, the nonportable PLT Scheme Web framework does not fit my requirements.

    Comment by apotheon — 10 September 2009 @ 09:21

  7. I have been through the same and sympathize with you, but what you ask for is currently not practical.

    Scheme suffers from way too many implementations and not enough libraries – it's just more fun for scheme hackers to build scheme core rather than scheme libs, and given each Scheme implementation actually have their own proprietary extensions, it is next to impossible for library writers to target all implementations.

    The solution here is not more standards – rather, some implementations must die. Really, is it that important the code must run on all possible including toy implementations?

    Natural selection force must exert on Scheme implementations to make that happen, and that means it is best to target the most vibrant scheme communities to shore up its infrastructure and let the other ones wither away. I have made my choice and I am building all of my plumbing code around PLT Scheme.

    PLT Scheme is portable to all the major OS platforms, and it is also R6RS compliant, plus having a ton of extensions that have helped blazed the trails for the would be standards, and a great platform to learn all about CS theories. From portability perspective, I am satisfied since I have it where I would do development (the only exception is the mobile platform – but that is not insurmountable), and many who have chosen PLT Scheme are also happy campers (that's why many web tutorials targeting PLT Scheme – even Paul Graham's Arc is built on PLT Scheme).

    It's okay if you are not convinced, but like you say, you might have to wait for current state to change, which might not, given R6RS is pretty much DOA. You might have to wait for R7RS and see if it takes off, and then see if the lib writers will port over to R7RS, and so on. You sure you want to spend your life in waiting?

    A shameless plug – I have a web framework in PLT Scheme if you do ever change your mind, take a look at http://weblambda.blogspot.com/2009/08/shp-02-and-jsmgr-01-http-01-are-now.html

    Cheers.

    Comment by YC — 15 September 2009 @ 01:06

  8. Scheme suffers from way too many implementations and not enough libraries – it's just more fun for scheme hackers to build scheme core rather than scheme libs, and given each Scheme implementation actually have their own proprietary extensions, it is next to impossible for library writers to target all implementations.

    I explained at length in the above SOB entry that I understand this about Scheme revisions prior to R6RS — and that this state of affairs is essentially a direct result of the way the standard was specified.

    Really, is it that important the code must run on all possible including toy implementations?

    It's that important that it run on all serious, standards compliant implementations of the same revision that don't have a special purpose that would preclude it. Just as I don't necessarily expect all implementations of a Linux-based operating system to run OpenOffice.org in all its glory, because some of them are embedded and some of them have licensing issues with the Java garbage needed to include some of the more outre features of OO.o, and I similarly don't expect Slackware 1.0 to run OO.o at all, there are times when I don't expect a given Scheme implementation to support a particular Web framework. On the other hand, just as one expects a recent version of any of the general purpose Linux distributions (Debian, Gentoo, Slackware, Ubuntu, Fedora, et cetera) to be able to run WEBrick and Rails, Catalyst, or Django, I expect a general purpose Scheme implementation to run a given Web framework written in/for that Scheme revision.

    Since R6RS specifies a standardized module system for libraries, though, I hope to see more portable libraries, and wondered if there was such a thing for Web development yet. That's all. If there isn't such a thing yet, that's too bad, but nobody has actually come out and said so. I don't see how trying to defend the non-portability of libraries across major implementations really helps anything, though, considering that fact is one of the reasons "serious" programmers tend to regard Scheme as an academic or "toy" language.

    Natural selection force must exert on Scheme implementations to make that happen, and that means it is best to target the most vibrant scheme communities to shore up its infrastructure and let the other ones wither away. I have made my choice and I am building all of my plumbing code around PLT Scheme.

    I'm rather annoyed with PLT Scheme, actually, because of its tendency to behave in a manner reminiscent of Microsoft — embracing and extending the standard in a manner that's incompatible with everything else under the sun. Now you're telling me you want to add the "extinguish" step to this. Great. I have a better plan:

    Why don't you get people in the PLT Scheme community to write libraries that are portable to other implementations that support the R6RS module standard?

    Of course, the way PLT Scheme seems dead-set on "adding value" to elements of the standard by tweaking them so they're technically conformant but incompatible with other implementations seems perfectly in keeping with its choice of license. Either the license or the module system in PLT Scheme seems designed to let code in, but prohibit any code from getting back out.

    Maybe I'm mistaken. From what I've seen, though, that really seems to be the case.

    From portability perspective, I am satisfied since I have it where I would do development

    "I'm satisfied with portability because it's all I use." Great. That's the opposite of portability: it's the open source implementation of vendor lock-in.

    It's okay if you are not convinced, but like you say, you might have to wait for current state to change, which might not, given R6RS is pretty much DOA. You might have to wait for R7RS and see if it takes off, and then see if the lib writers will port over to R7RS, and so on. You sure you want to spend your life in waiting?

    Maybe I'll end up spending my life with Common Lisp, once I'm done learning Scheme to a satisfactory level of competence. If the Scheme community never gets out of its implementation provincialism, that may be the only practical choice, anyway.

    I'm already using — and happy with — Ypsilon and Scheme 48 for now. I'm not going to change implementations away from the two I'm already using just to screw around with something that's not really central to what I'm doing with Scheme at this exact moment in time. I'm especially not inclined to do so if the future of Scheme library portability is really as bleak as you portray it.

    As for DOA . . . that's an interesting way to describe it. I guess, on the academic side, it might be. In terms of practical use, it seems to have some traction. Hell, Ypsilon was developed specifically because of a commercial need for concurrent garbage collection in a lightweight, but full-featured, R6RS implementation. That doesn't sound "DOA" to me.

    Comment by apotheon — 15 September 2009 @ 10:26

  9. > I'm rather annoyed with PLT Scheme, actually, because of its tendency to behave in a manner reminiscent of Microsoft — embracing and extending the standard in a manner that's incompatible with everything else under the sun.

    Hmm... how should I put it – allow me to try:

    1 – a good standard should come out from the practices, rather than trying to blaze the trail itself. That was how the CLTL2 and other standards are made – they gather all implementers and figure out what works for all 2 – without someone blazing the trail, you wouldn't have new standards. That's what PLT does 3 – PLT was very active in help developing the R6RS standards, and I am sure PLT will continue to do so with R7RS.

    > Now you're telling me you want to add the "extinguish" step to this. Great. I have a better plan:

    Did I sound that strong? Apparently I didn't do it justice:

    1 – the reason scheme lacks adoption is because it is the opposite of Perl, Python, Ruby, where there is one implementation and tons of libs 2 – the community focuses more on writing implementations than writing libs – part of this is due to the nature of the community. But it doens't help adoption 3 – the only way to get others to adopt scheme is to build more libraries, more documentations 4 – building libraries across implementations is still a very difficult exercise, even with R6RS, unless you want to build all of the platform specific extensions (such as OS interfaces, FFI, etc) yourself. It would take more experienced schemers than me 5 – to get to market earlier (and hence help enhance scheme) is to commit to one implementation and work out the kinks in the libraries themselves (and learn scheme at the same time), and possibly in due time port them to other serious platforms

    I choose to help, commit to a particular implementation, and get started. In time and with help I might be able to port my libs to other implementations.

    The mature schemes of course all have their own vibrant communities, and they'll survive the natural selection. For the ones that don't attract enough to sustain, we sure can use their time and talent to help grow the community in some capacity.

    > I don't see how trying to defend the non-portability of libraries across major implementations really helps anything, though, considering that fact is one of the reasons "serious" programmers tend to regard Scheme as an academic or "toy" language.

    I am not defending non-portability. I am saying – you can wait or you can get started.

    I also am trying to promote PLT as a good choice – given your sentiment, maybe not.

    Commit to one (or a couple), and get it flourished first.

    > Why don't you get people in the PLT Scheme community to write libraries that are portable to other implementations that support the R6RS module standard?

    I read it somewhere, maybe Paul Graham, and the author defends lisp's multiple implementations and is not too concerned about fragmentation as it can be solved with compatibility layers.

    In CL, there are compat layers to Allegro and others that proved his theory.

    While such compatibility layers for PLT do not exist yet, it can be built provided enough effort can be dedicated to it. Would you like to take up the mantle for such cause? With one effort you can learn scheme to expert level, and you can unlock all PLT specific code.

    > As for DOA . . . that's an interesting way to describe it. I guess, on the academic side, it might be. In terms of practical use, it seems to have some traction. Hell, Ypsilon was developed specifically because of a commercial need for concurrent garbage collection in a lightweight, but full-featured, R6RS implementation. That doesn't sound "DOA" to me.

    Well – R7RS is already under way – it's too soon if everyone was happy with R6RS. But maybe R7RS would stay in development hell so R6RS gets some field time. We'll see.

    > I'm already using — and happy with — Ypsilon and Scheme 48 for now.

    Then commit to them and help Ypsilon and Scheme 48 grow. Or if you want, you can help me unlock my code over to Ypsilon and Scheme 48 – I'll work with you. There are so many ways to help, but it's important to commit and get going.

    Cheers.

    Comment by YC — 15 September 2009 @ 01:34

  10. Speaking of compatibility layers, I'm currently reading The Art of the MetaObject Protocol. The MOP was developed as a way to subclass the behavior of the OOP system of the language itself, mostly so that code from OO implementations other than CLOS could be ported to CLOS with minimal pain and without impacting other CLOS code. Something like that for Scheme's library implementations might be useful, I'd guess.

    Comment by Chip Camden — 15 September 2009 @ 02:11

  11. YC:

    without someone blazing the trail, you wouldn't have new standards. That's what PLT does

    That's great. That's not all PLT does, though. From what I've seen, it also seems to prefer non-portable libraries, even for R6RS libraries.

    building libraries across implementations is still a very difficult exercise, even with R6RS, unless you want to build all of the platform specific extensions (such as OS interfaces, FFI, etc) yourself.

    . . . or maybe put them in portable libraries. One would think having a de facto standard library (maybe the Scheme version of Boost, in terms of its widespread acceptance and usability despite a myriad of C++ implementations) would be something people would want, rather than to try to tie everything directly to the language implementation so that portability of libraries doesn't become effectively impossible.

    to get to market earlier (and hence help enhance scheme) is to commit to one implementation and work out the kinks in the libraries themselves (and learn scheme at the same time), and possibly in due time port them to other serious platforms

    I don't see much value in getting "to market" with something that's going to be largely ignored because it's so impractical.

    I am not defending non-portability.

    Why, then, did you start acting like I said something wrong when I described exactly the same problems with library portability you described and asked about a portable Web development framework? You basically told me my preference was wrong, and I should use PLT Scheme for Web development.

    Commit to one (or a couple), and get it flourished first.

    I'd rather commit to portable library development, just as soon as I know Scheme well enough to do some useful work in that area. Telling me to favor non-portable development in a specific implementation — even with the ultimate goal of porting libraries to create parallel non-portable versions of the same libraries so the libraries can then (laughably) be called "portable" — isn't really going to suit that preference.

    I read it somewhere, maybe Paul Graham, and the author defends lisp's multiple implementations and is not too concerned about fragmentation as it can be solved with compatibility layers.

    I love the fact there are so many implementations. I just don't like the fact nobody seems to want to write portable libraries, even though there's now a way to write portable libraries defined in the standard.

    While such compatibility layers for PLT do not exist yet, it can be built provided enough effort can be dedicated to it. Would you like to take up the mantle for such cause? With one effort you can learn scheme to expert level, and you can unlock all PLT specific code.

    Maybe some day. Maybe not. For now, I'm still trying to find time to actually learn some Scheme beyond basic syntax. That's why I was hoping there'd be something akin to the Web dev tutorial for CL, suitable for use with Scheme, that doesn't require me to install yet another Scheme implementation (and encourage me to change editors).

    Well – R7RS is already under way – it's too soon if everyone was happy with R6RS. But maybe R7RS would stay in development hell so R6RS gets some field time. We'll see.

    I get the impression, based on the talk about two tracks for future Scheme development, that R7RS is basically going to be R5RS++ and R6RS++ as parallel dialects. That doesn't make me think that using R6RS is pointless. Why not just stop using R5RS conventions, too? When you answer that for yourself, maybe you'll understand why I think your comments about R6RS aren't very helpful.

    Then commit to them and help Ypsilon and Scheme 48 grow.

    Right now, I'm committed to learning Scheme. You seem to have this bizarre idea that I can just start hacking the core of Scheme, or writing widely-used libraries, or something like that. I'm a beginner at Scheme. I wouldn't even call myself marginally competent yet.

    Maybe some day. Right now, I'm not ready.

    Comment by apotheon — 15 September 2009 @ 03:06

  12. Sigh... apparently I just wasted your time and mine.

    This is my last response, not to waste more of your time, but just to clarify that I was hoping to tell you to get going and don't wait on portable libs and let it stopping you from learning scheme. As to the rest – maybe we'll discuss them again once you pick up scheme.

    If I missed the mark it's my bad.

    Cheers.

    Comment by YC — 15 September 2009 @ 03:46

  13. No worries there. I'm not going to stop learning Scheme. I want to learn Scheme — and, even if I didn't care about Scheme per se, I'd learn it anyway because it's the example language used by SICP to teach computer science concepts. I was just hoping that a decent Web development tutorial that didn't specifically rely on non-portable libraries might exist to give me a bit of a boost in more quickly achieving some level of competence with Scheme.

    Comment by apotheon — 15 September 2009 @ 04:35

  14. I feel your pain. I've been going through the academic part of learning Scheme: SICP, Little Schemer, Seasoned Schemer, etc. And now I'm trying to get my footing in the world of "useful/practical" Scheme applications. I just wrote a shell script to help me with merging conflicts in Scheme. My first "useful" Scheme program that I use at a real job!

    These sites may help you out a bit.

    This site provides a repository of cross-scheme APIs: http://snow.iro.umontreal.ca/

    And here is a portable multi-purpose Scheme library: http://people.csail.mit.edu/jaffer/SLIB

    Comment by Mark — 28 October 2009 @ 03:05

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

All original content Copyright Chad Perrin: Distributed under the terms of the Open Works License