This is Paloma, a program to manage a database of digital music files (typically in MP3 form, although not necessarily) and facilitate their retrieval and playback in interesting ways. It's intended to replace a traditional CD player/changer system; without too much effort, you can rip your entire CD collection and store it on your hard disk for instant random access. The original version of Paloma was developed for Linux and gtk. As of Paloma version 2.0, it has now been rewritten using Python and wxWidgets. This means it should be more portable to other platforms, including Windows and Mac, although the development emphasis is still on Linux. This program is distributed under the terms of the GNU General Public License. It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA To contact the original author of this program, write to: David Rose 1957 Chilton Dr. Glendale, CA 91201 USA paloma@notspam.com Replace "notspam" with "ddrose" in the above address. Note that this email address *might* have a spam filter installed (it doesn't right now, but I reserve the right to install one in the future). If you get a bounce message, read the text of the bounce message carefully for instructions regarding the actual email address to write to. You can obtain the latest copy of this software at http://paloma.sourceforge.net . You will need: * Python. You may already have it installed with your Linux distribution. Type 'python -V' on the command line to see if it is installed (this will also report the version number you have installed). I am sure that you will need at least version 2.0, but it might be that you actually need 2.2 or better (let me know if you find out whether this is or isn't true). Paloma was developed on Python versions 2.2.1 and 2.2.3. You can download the latest version of Python from http://www.python.org/ . Note that it is possible to have multiple versions of Python installed on your system at once and that they generally play together nicely. * wxWidgets, version 2.5.1 or better, and the associated wxPython wrappers. You can get this, in source form or as precompiled binaries, from http://www.wxpython.org. * Either PostgreSQL or MySQL, as a SQL database backend. There are pros and cons of both, but I consider them pretty interchangeable. You can get Postgres at http://www.postgresql.org/ and MySQL at http://www.mysql.com/ . You may also already have one of these installed. I don't think there are any major version requirements here; just get the latest version. * The appropriate Python interface to your database of choice. For PostgreSQL, you can use either PoPy, which you can find at http://sourceforge.net/projects/popy/ , or pyPgSQL, at http://pypgsql.sourceforge.net/ . For MySQL, use MySQLdb, at http://sourceforge.net/projects/mysql-python/ . It's awfully nice to have: * CDDB.py. This is a Python interface for reading the track information from the CD-Rom and downloading the track titles from the Internet (via CDDB or FreeDB). It's not strictly necessary: you can run without it, but you won't be able to import CD's. You can find it at http://cddb-py.sourceforge.net/ . * Some command-line based cd-to-digital audio ripping program. Again, this is only necessary if you want to load CD's. I recommend cdda paranoia, at http://www.xiph.org/paranoia/ , but other programs like cdda2wav, for instance (at ftp://ftp.gwdg.de/pub/linux/misc/cdda2wav/ ) should work also. You may already have one or both of these installed in a typical Linux installation. * A command-line based CD player is handy, too, for playing tracks on CD's before you encode them. You want one that won't exit while the CD is still playing, and that will stop the CD when it is killed. I haven't found one that meets both of these requirements, but dcd comes close (it just doesn't stop playing when it's killed). You can find it at http://dcdplayer.sourceforge.net/ . * Some command-line based MP3 encoder (assuming you want to import MP3's from non-MP3 sources, like CD's). The most popular one I know of is LAME at http://lame.sourceforge.net/ . There are dozens of other encoders available as well; let me know your success or failure stories. * Some command-line based MP3 decoder/player. Of course, you only need this if you plan to store mp3's in your database. mpg123 seems to be a good choice (I recommend at least version 0.59q, as it reports the total length of the file, which Paloma likes to know), at http://www.mpg123.de/ , but anything that can be launched from the command line should work, including GUI players like Xmms at http://www.xmms.org/ . There are dozens of players available; let me know if you try something and it works or it doesn't. * A command-line base ID3 tag editor / reader is handy if you want to record the song names, artist names, etc. in the MP3 files themselves, or to get the names out when importing a directory of MP3 files the first time. Paloma doesn't do anything with the tag information (it's all already stored in the database), but other external programs might, and it's handy to have this information built into the files. There are many such programs; I use id3ed, which you can find at http://www.azstarnet.com/~donut/programs/id3ed.html . As with all of my suggestions for external programs, you may use this program or a different one of your choosing. * If you intend to store your music in Ogg Vorbis format, or some other file format, you will need a command-line program to convert sound files to this format, as well as a command-line program to play them and/or tag them. Since I haven't experimented with this file format myself, I don't have any specific recommendations, but if anyone else has any suggestions I'll be happy to record them here! INSTALLATION First, make sure you have the requirements listed above (Python, wxPython, and PostgreSQL or MySQL) installed on your machine. Then run the following command, as root: python setup.py install Note that if you have multiple versions of Python installed, you may need to specify which particular version you wish to use to install Paloma, e.g.: /usr/bin/python2.2 setup.py install This script will compile the C-based support libraries used by Paloma, and then copy all of the appropriate files into your Python site-packages directory, usually something like /usr/lib/python2.2/site-packages. Finally, it will copy the scripts "paloma", "sfserver", and "ecserver" into /usr/bin or /usr/local/bin (according to how your version of Python is installed). ONCE PALOMA IS INSTALLED Once you have done the above, you should be able to run Paloma from the command line: paloma And, if all goes well, it should reply something like this: Unable to connect via PostgreSQL: FATAL: database "paloma" does not exist Unable to connect to database paloma. Because you haven't created the database yet. (This is the Postgres error message. If you are running with MySQL, you will see a slightly different error message.) You must create a database named 'paloma' before you can go any further. Do this now. Generally, this will be as simple as starting up your SQL monitor (with a command like 'psql' or 'mysql') and typing the SQL command: create database paloma; If you get some sort of permission objection to this command, you need to use your SQL administration tools to give yourself permission to create databases, or you can just execute the command as the SQL administrator (probably 'postgres', for PostgreSQL, or 'root', for MySQL). A detailed explanation of how to do this is outside the scope of this document; you'll have to browse through the documentation that shipped with your database software. Once you have the database created, launch paloma again. This time, you should get a window with an image of a dove in it, and a menu bar. Welcome to Paloma! CONFIGURING I tried to set Paloma up so that, as shipped, it should require minimal configuration for a typical system. However, you will almost certainly need to set at least a few options. Unfortunately, there is not yet an online help for the configuration window, and it can be quite mysterious. I will attempt to explain the options here as well as I can. You will probably need to set the following things: * Shell commands to play the various kinds of sound files. This includes playing tracks directly from the CD-Rom. * Shell commands to convert between the various kinds of sound files. This includes the command to rip tracks from the CD-Rom to WAV format. * The directories into which WAV files and MP3 files will be stored, after extracting from the CD-Rom. Start up Paloma, and pick "Configure" from the "File" menu. You'll get a window with a traditional tabbed configure page, featuring several pages available under different tabs. Tab 1: File types On the File types page, you can define the various kinds of sound files that Paloma knows about, as well as define commands to play them and convert between them. As shipped, Paloma knows about WAV files and MP3 files, plus a special-case "file" for playing directly from the CD-Rom. Pick a file type from the combo box at the top to set the commands to play or convert from that kind of sound file. Some reasonable defaults are filled in for you, but you might need to adjust them according to the particular command-line utilities you have installed. In particular, you'll want to check the MP3-play command, the CD-to-WAV convert command, and the WAV-to-MP3 convert command. (If you plan to rip directly from CD to MP3, you won't need the WAV commands, but you will need a CD-to-MP3 convert command.) (You should also read the explanation below about play response, verify command, and verify response. If you don't understand these fields, you can leave them blank, but you'll get better control from Paloma if you can find meaningful values to put in here.) You can also define a new soundfile type of your choosing on this page by backspacing over the name in the combo box at the top, entering the name of the new type, and clicking "Create type". The various fields you can set here are: Play command: This is the shell command to play this particular type of soundfile. It can be any shell pipeline. The following variables are available for use: $sound_device - The sound device to contact, typically /dev/dsp. $filename - The sound file to play. $cdrom_device - For the CD-Rom entry, the particular CD-Rom device. $track - For the CD-Rom entry, the track number on the CD. $start_time - The number of elapsed seconds in the soundfile at which to start playing. $end_time - The number of elapsed seconds in the soundfile at which to stop playing. $net_time - The total number of seconds to play. ( = $end_time - $start_time) $total_time - The total number of seconds Paloma believes the soundfile to contain. $start_frame - The first frame number to play. $end_frame - The last frame number to play. $num_frames - The number of frames to play ( = $end_frame - $start_frame) $total_frames - The total number of frames Paloma believes the file to contain. $start_percent - The percentage into the soundfile at which to start playing. $end_percent - The percentage into the soundfile at which to stop playing. In the above, $start_frame etc. is just another way to express the same concept of $start_time etc., but in units of frames instead of time. Precisely what is meant by a 'frame' is up to the soundfile player, and Paloma can only know the frame counts if the verify command (described below) retrieved that information. If your soundfile player cannot start and stop the playback at random points within the soundfile, simply do not include any of the start/stop variables in the command string, and Paloma will recognize this and only allow playing of the soundfile from the beginning to the end. Play response: This is an optional field that defines a line of text to look for in the soundfile player's output. Some soundfile players report the current position through the file as they are playing, so this can be used to parse that string and update the slider in the Paloma window. If your soundfile player does not report its current position, leave this field blank. Note: the default values filled for mp3 play (and verify) response are based on mpg123 version 0.59q and later versions. If you have an earlier version of mpg123, you will need to erase these fields and leave them blank, as the output of these earlier versions of mpg123 is not directly useful to Paloma. The following variable names are recognized, and are interpreted as values of the following meanings: $current_frame - The current frame number in the file. $frames_remaining - The number of frames before the end of the file. $total_frames - The total number of frames in the file. $current_time - The current number of seconds elapsed in playback, (or a time in the form MM:SS or HH:MM:SS). $time_remaining - The number of seconds remaining before the end of the file (also in the form MM:SS or HH:MM:SS). $total_time - The total length of the file in seconds, or MM:SS or HH:MM:SS. $current_pos - The relative position in the file as a decimal in the range [0 .. 1]. $current_percent - The relative position in the file in percent. Verify command: This is an optional field. Before playing a soundfile, Paloma may try to verify that the soundfile exists, or obtain information about the soundfile like the total length in seconds or the total number of frames. (Paloma must know the total number of frames in order to fill the $start_frame/$end_frame variables correctly, above.) If this field is nonempty, it contains the shell command to execute immediately prior to making a soundfile available for playback. This command should presumably read the soundfile header information and report information about the file. If the program exits with a failure status (an exit status of nonzero), the file is deemed unplayable. Verify response: Like play_response, this is the line of text to look for in the output from the verify_command. The same variables as those valid for play_response are recognized and interpreted. It is particularly important for the verify response to extract the length of the soundfile, if possible--this allows the song length to be recorded in the database when importing a directory of songs. Write tag command: This command is used to write song information into the soundfile itself, for instance id3 tags on MP3 files. Paloma does not care about these tags, since it has all of that information in the database, but an external program such as a jukebox may read it and take advantage of it. At the present, this command is only invoked when you choose the option "Tag songs" from the main menu or from a playlist menu. Eventually, Paloma may have an option to automatically tag songs as they are imported. The variables that may be included here are similar to those available in the Export dialog within Paloma. They include: $name - song name $performer, $writer (first last) $performerlf, $writerlf (last, first) $cdtitle, $cdperformer, $cdwriter, $cdperformerlf, $cdwriterlf $track - track number Tag encoding: This specifies the encoding system to use for writing international characters (non-ASCII accented letters) into the tags for this kind of soundfile. It is the same sort of thing as the encodings specified under the Encodings tab. See Tab 4: Encodings, below. Read tag command: This is the command invoked to read the tags from existing soundfiles. It is only used when importing a directory of songs into Paloma. This should be the name of a program that will output tag information to standard output. Read tag response: This field describes how to interpret the output of the read tag command, above. It is essential to being able to intepret tags, since invoking the command is useless without being able to understand its output. It should consist of one or more lines of text that might be output by the tag command, with variable references standing in for values that will be extracted from the line, similar to the verify and play responses, above. You may use any variables you like; the values that these match will all be available to the user after scanning a directory in the import dialog. However, the variable names from the write tag command, above, are expected and treated as their defined meaning. Predict-length expression: This field contains an algebraic expression which, if nonempty, Paloma will use to estimate the expected file size of the resulting soundfile when ripping, based on the length of the track in seconds. This is only used to update the progress bar incrementally while the track is ripping. Presently, the only variable supported in this expression is $time (although for consistency with the play command, this may also be written as $total_time). For WAV files at 44100 Hz, 16-bit stereo (CD-quality), the formula should be something like ($time * 44100 * 2 * 2). For MP3 files at 128 kbps, the formula should be ($time * 128 * 1024 / 8). Filename extensions: This is a space-separated list of the filename extensions that are likely to indicate this particular kind of soundfile. This is optional; Paloma stores the known file type of a given soundfile in the database, but it helps to know the extension in order to guess the type when importing soundfiles, for instance. To convert to type: This is a special field. It consists of several commands, one command for each kind of soundfile this soundfile type can be converted to. Pick the soundfile type to convert to, and fill in the command to do the conversion. You don't need to fill in a command for each combination, just the ones you care about. (In fact, most combinations are blank, as shipped. That's fine.) This is the field that holds two very important commands, which you might miss: (1) the command to rip CD's to WAV files (convert CD-Rom to WAV), and (2) the command to encode WAV files to MP3 (convert WAV to MP3). There's also a command to convert MP3's back to WAV's, handy for burning CD's from a given playlist. You have the following variables in the conversion commands: $cdrom_device - For the CD-Rom entry, the particular CD-Rom device. $track - For the CD-Rom entry, the track number on the CD. $from - For a non-cdrom entry, the filename to read. $to - The filename to write. You should put quotation marks around the filenames "$from" and "$to" on the command to protect filenames that may contains spaces or other special characters from the shell. If you plan to rip CD's directly to MP3 form to save on temporary disk space, then instead of having a CD-to-WAV conversion command, you'll need a CD-to-MP3 conversion command (convert CD-Rom to MP3) which will be a shell pipeline to rip and convert a single track. It might look something like this (depending on the utilities you have installed): cdparanoia -d $cdrom_device $track - | lame - $to (In this case, you'll also need to set the field "Load from CD-Rom as file type" under the CD-Rom tab, below, to MP3.) Tab 2: Volumes Paloma uses volumes to manage collections of files. A volume is simply the root of some directory hierarchy. In the current release of Paloma, a volume must be some local (or NFS-mounted) directory, although future versions of Paloma may support URL's and other foreign-locator references as volume definitions. As shipped, there are two volumes defined. One, "Loading area", is the directory into which WAV files are stored as they are read from CD-Roms. This is just a temporary holding area, as they will soon be encoded into MP3 form. When they are, they will be stored in the second pre-defined volume, "MP3 storage". As with the soundfiles, above, you can define any new volumes that suit you by backspacing over the description, entering your own description string, and clicking "Create volume". The minimum thing you need to give for each volume is its definition--the location of the volume on your disk (and you must then create that directory yourself). The other three fields are all about buckets, which is just a handy way to keep your directories from growing obscenely large and making it difficult to back up your files. Buckets are only useful when you will be using Paloma to put files into the volume (e.g. by ripping tracks from CD's); volumes that are already filled up with files from other sources will not use buckets. The various fields you can set here are: Definition: This is the full directory name of the root of the volume. It must, currently, be either a local directory name or some mounted directory in your filesystem. Bucket size (MB): This is the maximum number of megabytes to put in any one bucket. If this number is not zero, when Paloma puts files into this volume it will automatically create subdirectories under the volume root, one for each bucket, and it will not fill up any one bucket more than the indicated number of megabytes. The default for "MP3 storage" is 650, which allows the buckets to be backed up onto recordable CD's. You can change this to suit the size of your backup medium; for instance, set it to 100 to make it easier to back up to Zip disks. Once a bucket is filled, it will never be changed again by Paloma (except to remove files), so CD-R's are an ideal backup medium. Current bucket number: This is the current bucket number Paloma believes it is working on. It will automatically increment as buckets fill. Normally, you would start it counting at 0 or 1, whichever suits you. Bucket subdirectory name: If bucket size, above, is not zero, you should have some string here that includes the variable "$bucket". This will be the name of the subdirectory that is automatically created for each bucket, where the variable $bucket is replaced with the bucket number (filled to three digits). This string should not contain any slashes. If bucket size is zero you should leave this blank, and buckets will not be created. Tab 3: CD-Rom The CD-Rom page describes a few things particular to reading from CD-Rom devices. Most of these you probably won't need to set; the defaults should be sufficient. The various fields you can set here are: Soundfile type of CD-Rom device: This simply indicates to Paloma which of the soundfile types listed under the "File types" tab is the one that indicates the CD-Rom device. Normally, this will be the one called "CD-Rom", although if you're perverse you could of course set it to any of the others. You probably shouldn't, though. Note that the CD-Rom device has a few exceptions for interpreting shell commands: the variables $cdrom_device and $track are used instead of $filename, for instance. Load from CD-Rom as file type: This is the file type that your CD-ripping program produces, and it also tells Paloma where to look for the CD-ripping command. When you start to rip a CD, Paloma excutes the convert command (from the File types page) that converts from the CD-Rom type to the type indicated here. Normally, this will be WAV files. Note also that you can set this to MP3 files (and provide a suitable CD-to-MP3 conversion command in the File Types tab) to rip directly from CD to MP3 form, if you're short on disk space. Store loaded files in volume: This is the volume into which the files as loaded directly from the CD will be stored, before they are encoded into their final form. It's thus just a temporary holding area. Encode from loaded type to file type: This is the file type into which the WAV files loaded from the CD-Rom should ultimately be converted, by the background encoding process. Normally, this will be MP3. Store encoded files in volume: The location into which to drop the thus-encoded files. This will be the final resting place of the songs you rip from your CD's. Tab 3: Encodings If you plan never to import any songs or artists with international characters (or accent marks) in their names, then you can ignore this tab. Even if you do plan to use international characters from time to time, you can probably leave the values here set to their defaults, but an advanced user may want to tweak these. The de facto standard for representing alphabetic characters in the world today is ASCII, but that only represents the uppercase and lowercase English (Roman) alphabet, plus digits and a few punctuation marks. If you want to use any other special characters, or use accent marks on any of your letters, then you need to use some other encoding system. ASCII is technically a seven-bit code, but it is stored in eight bits with one bit unused. Probably the next most common standard is Latin-1, sometimes called iso8859. This coding system improves upon ASCII by adding the accented letters that occur in the most common Western European languages. Latin-1 takes advantage of the extra bit unused by ASCII, and is thus an eight-bit code, and is represented easily in today's eight-bit databases. However, there are some characters that cannot be represented in Latin-1. (There do exist other eight-bit conventions, like Latin-2, Latin-3, etc., and a whole slew of incompatible Windows conventions, which all choose a slightly different set of characters to include in the eight-bit space available. Paloma doesn't offer any of those since they can all be represented with the following system, UTF-8.) Finally, the Unicode standard has defined a 16-bit code that defines almost all characters that appear in human languages, including the many thousands of characters in East Asian languages. Since this is a 16-bit code, it cannot easily be represented in our current file systems and databases of the day, which all store streams of eight-bit data, but there are various ways for representing 16-bit data using combinations of eight-bit bytes. One popular system is called UTF-8. It has the nice property that normal ASCII characters are represented the same way they are in ordinary ASCII encodings (and in Latin-1), while the international characters beyond ASCII are represented using special two- or three-byte combinations for each character (as opposed to the one byte that Latin-1 uses). In this way, the entire set of 16-bit codes available in Unicode may be represented in an eight-bit stream. It is up to you to specify which system is to be used to represent international characters in filenames, within the database, and in the tags written to and read from the various kinds of soundfiles. Paloma doesn't care, and will happily use any of the three listed (others are also possible--if you have a real need for a different encoding system, let me know). The only reason to choose one encoding system over another is the interaction of systems other than Paloma. For instance, the operating system may have its own restrictions as to what kinds of characters may be written into filenames; or you may want to use an MP3 player that assumes tags are encoded using Latin-1. ADVANCED INSTALLATION You're done with the basic stuff. You can now start importing CD's and directories into your Paloma database and see what you think of it. But the power user may want to know more about Paloma's optional daemon processes. There are two: sfserver This process supervises the playing of sound files: WAV and MP3 files, as well as audio directly from the CD-Rom (or for that matter, any kinds of sound files you defined on the "Sound files" tab, above). It listens on a socket for a connection from a Paloma process, and manages the actual interface to the sound device. Running the sound file server as a separate program has a few advantages: * It is not necessary to run Paloma on the same machine as the one with the sound card. Maybe one computer is connected to the speakers in your living room, while you're actually working on a different one; you can use Paloma to contact the machine that actually has the sound card you want to play through. * You can have multiple Paloma processes running on the same (or different) machines, using the same sound card, and sfserver will negotiate the contention for the sound card. If you start a sound file playing in one Paloma process, it can automatically interrupt the one playing in another process. * You can run the sfserver process at a super-high priority, without having to make it suid root, so that your music will not be interrupted by CPU-intensive but less time-critical things such as compiles happening in the background :). And Paloma, the GUI, can run at the normal priority, so as not to unnecessarily bog down your machine. If you don't have an sfserver process running, Paloma will play sound files directly. ecserver This is the "encoding server." This process waits in the background for new WAV files to be loaded from CD. As soon as it sees one (it checks every five minutes), it begins encoding it to MP3 form. It also handles other kinds of batch conversion requests that are issued from the "Batch convert song files" menu option from a playlist. You might have multiple ecservers running on the same database, which is particularly useful if you have multiple CPU's in one PC, or multiple PC's on the same LAN. In this way, you can distribute the conversion process, so that converting large directories of files don't have to be bottlenecked by one CPU. Running a standalone ecserver process also allows you to shut down Paloma (and even log off) without interrupting the batch conversion process that might still be underway. And finally, you can choose to run ecserver at a lower priority than your other processes, so that it uses only spare CPU cycles to do its work, without impacting your other actitivies. If you don't have an ecserver process running, Paloma will do batch conversions directly, but if you exit Paloma the batch conversions will be interrupted too. Normally, you will want to run sfserver at either a normal priority or a super-high priority, depending on the relative importance you put on having uninterrupted music; while you might want to run ecserver at a very low priority. If you choose to run these services, you should configure your machine to start them at boot time. How you do this depends on your installation, and isn't part of this document. A WORD ABOUT COPYRIGHTS AND COPYRIGHT VIOLATIONS Paloma is intended to facilitate the management of your own music and uncopyrighted music freely available on the net--music which you legally own the rights to play. It is not intended to facilitate copyright violations, although like any music management tool, it could be misused in this way. It is ethical (and probably legal) to copy your own, copyrighted CD's to your hard disk and play them from your computer, for as long as you also still own the original CD's. The ethics and legality are less clear in any other situation. As a general rule of thumb, you are honoring the spirit of the copyright law as long as you are not enabling a piece of copyrighted music to be played by more than one person simultaneously. If you borrow a friend's CD to add it to your database, as soon as you return the CD to your friend and still have the music on your PC, you are in violation of copyright. If you rip your CD collection and then sell or give away your CD's, you are in violation of copyright. If you let anyone download MP3's from your database, or you download copyrighted MP3's from someone else, you are in violation of copyright. (Strictly speaking, you are in violation of the spirit of copyright. Whether you are in violation of the actual law itself is not so clear.) Whether you should care about this or not is of course totally up to you. But consider this: the recording industry has been raising an enormous stink lately over their perception of the use of an open format like MP3 to pirate music, and could conceivably arrange legislation forbidding or restricting the use of MP3's for good, legitimate purposes, like putting all your own music into a relational database. I don't expect it to happen, but there's no need to make things worse. Don't give them any ammunition.