Stick Theory: How To Use x264 Like A Pro, Only Pro’s Can’t Legally Use It For Production Work
Just as String Theory attempts to govern the universe, Stick Theory governs how x264 works. Stick Theory is the theory that when choosing x264 settings, even if you have no idea what anything does, as long as your keep throwing shit at a wall, you’ll see what sticks, resulting in quality video.
A trend I notice is that very few people can really use x264 properly. Everyone expects wonderful magical GUI’s and automated shenanigans where you just set a preset and leave it, and your video comes out looking reasonably non-crap. The thing many people don’t know is that CLI is good for you, and not as scary as one might think. Additionally, x264 has presets now, and soon, a –device flag for incredibly simple encodes for PS3 or Windows Mobile or whatever else. This makes basic x264 usage really simple. Just read x264 –help, or if you want more comprehensive info, –longhelp or even –fullhelp.
I lurk in a lot of IRC channels. More and more often now, I am seeing people asking in places like #Archlinux how to encode. The channel is for linux discussion, not encoding. At the same time, the guys in #x264 don’t really like handing out tailored settings for every random lazy guy that drops into the channel. Spending two minutes on a quick google search will give you most info you want to see. http://mewiki.project357.com/wiki/X264_Settings should be one of the first results, and it has more info than pretty much everywhere else on the entire internet besides for possibly the musty halls of Darkhold, or inside Loren Merritt’s head.
I also see a lot of users calling Random Distro X crappy just because it doesn’t provide a crappy tool like Handbrake to transcode their porn with. The thing is, x264 is really easy to use. It’s simple, can be installed or compiled in seconds, and with the preset system you can do pretty decent encodes as long as your source is good. There is no real need for a GUI or for people to think encoding is hard. x264 is practically made for the lazy man now. Stop filling up my 119 IRC channels with useless shit about why you suck at encoding people.
The other problem with x264 users of course is that they often don’t know what different settings do, and mess with them anyway. This is where Stick Theory comes in. While throwing random shit at a wall DOES eventually give you a good result, what is significantly faster and more informed is to learn what common x264 settings do, especially things like the vbv-buffer, crf, aq, psy-rdo, trellis, bframes, b-pyramid, subme, motion estimation, rc-lookahead, and a few others. The above linked wiki article is a very good repo of x264 info mostly written by some very talented encoders (look Dae, I called you talented!)
Knowing what you’re doing will let you write out a full x264 command string in less than a minute to have a very well tuned encode, tailored for your content, resulting in VERY nice results. Of course this isn’t strictly necessary, but knowing it always helps. That does however sound tedious. Luckily, with x264, you can use both the presets AND the main flags, allowing you to choose a preset, autotune it with the –tune flag, and then override as necessary for a very easy yet customised encode string. And all in the space of a minute.
Now, stop being lazy and get off my lawn and out of my IRC channel. x264 does all the work for you, it’s MAGIC I say! (Some might argue that the AI in things like AQ and CRF is sentient, but Dark_Shikari assures me it isn’t, yet)
Linux Was Clearly Never Made For Audiophiles
I am not an angry man. I have absolutely no discretion and I am almost always correct in whatever it is I am saying (disregard all previous posts here and that statement remains mostly true) but rarely do I really lash out, until I come across the topic of sound in Linux. It’s not like the situation is bad enough with sound servers (OSSv4 is awesome but doesn’t support the card I want, everything else can go burn for all I care) but what REALLY shits me off is that lack of good media players.
My current media player? mplayer-uau. It has no support for tags in FLAC or TTA but otherwise it’s by far the most stable player I have, and it’s not even that inconvenient to use it from CLI either. I recently picked up an album by one of my favourite artists, Alstroemeria Records, and was a little sad to find it so flat in mplayer, because I’m pretty damn sure it sounds excellent on the Cambridge Azur 650C. I decided to try some Qt based media players and see if any of them could make me happier, but I figured a player wasn’t going to make much difference, I really hate equalisers but something told me to try anyway.
My first step was Amarok2. My last experience with Amarok2 left me with an intense hatred of the thing. It continually crashed, it’s media collection and tag reading/writing system is worse than WMP7′s (yes really), and it has on more than one occasion permanently deleted tracks from my harddisk. This sorta gave me a bit of “don’t go near amarok2 ever again” vibe but I decided I had all my shit backed up and it would be worth a shot. For once it didn’t crash, and has only done so once in the past 72 hours. It seems to be fucking up sending things to last.fm but otherwise fairly ok. Seeking is shitty in FLAC and TTA somewhat refuses to seek at all. I did try using different phonon backends and none of them made me any happier. mplayer, xine, and gstreamer for anyone that cares. Seeing as this gave me no advantage over mplayer, I moved on.
My next try was Clementine, the Qt4 fork of Amarok 1.4. The UI was familiar enough but the complete lack of features pissed me off, so I didn’t really get much real testing done here. I did take the opportunity to try configuring a library in it to which is stalled for 20 minutes and eventually crashed. After this somewhat half-assed attempt at a media player, I decided to stick with the theme of doing half a job and tried Prismriver. I actually intended to use this name myself elsewhere and may still do (that or Alstroemeria, so I can name something after what I intend to use it for) but regardless, my first 4 attempts at compiling the thing failed. Eventually I got it to go, and had some nice segfaults. Not even able to test the player’s fabled cuesheet reading which is a feature I really need when I’m too lazy to split and tag my stuff manually.
7 shitty players later, I realised I had qmpdclient and ncmpc installed so I promptly decided that GUI’s for media players in Linux are all terrible and went with CLI. Working fine for the moment. ncurses is an excellent invention and makes the best GUI’s I guess. Before people ask why I didn’t try GTK+, it’s because GTK is a horrible piece of crap and every developer that uses it is a retard that deserves absolutely no recognition for their work which nobody should even use anyway. Qt has Phonon too, which I bet could be implemented into GTK but that would give people a reason to keep using it, and that would be bad.
ITP: I make my own media player using phonon and a backend that isn’t a load of cancer inducing shit. Except it would take forever because I suck at anything not on embedded chips.
On a side note:
Dear Open Sound System Developers,
Please support the ASUS Xonar Essence STX in OSSv4 so that I may use it with my sexy amp and DAC and headphones and be a huge audiophile dickwad.
Love,
Emess
Boot Optimisation Is Entirely Useless When You Reboot Once Or Twice A Year
A few months ago, I spent a weekend optimising the shit out of my system to the point where I could press the power on button and be fully logged into KDE with all my shit open in 8 seconds flat. It seemed like a cool idea at the time as I didn’t have an SSD and that was FuckingFast.png, however I didn’t really think about the fact that I reboot maybe 5 times a year (I totally hotload my kernel: don’t do this) and the fact that fsck likes to run every 90 days.
Enter KDE and nvidia collectively shitting themselves this morning somehow, and Xorg eating my CPU like it was ice cream in a heatwave. I managed to SSH in from another PC to get KDE to cleanly close down (didn’t actually work it seems) and then reboot the PC. At this point I had to leave my house in 5 minutes and needed to check something really important rather quickly. Then I got fscked. Hard. See, it never occurred to me that fsck would need to run on my enormous LVM, nor how long that would take.
I ended up checking what I needed to on my phone and then coming home a few hours later to see fsck at about 80%. Eventually it finished it’s shit and I got logged in to find my KDE session opening shit I closed months ago, and about 6 hours of YATTA work lost (maybe not) as it was open when the system crapped a brick. I still have no idea what happened, nor why YATTA was using 37% of my RAM yet still slow as hell, and where all my swap went, but at least I know that boot optimisation is entirely useless. I am totally getting an SSD in a few weeks to bring this down to sub 5 seconds.
Lucky For My Siblings That Homicide Is Illegal
This will be a short post, as I don’t trust myself to write anything that properly conveys my rage without murdering a member of my family. I like to download my TV shows. My sister likes to watch TV shows. Occasionally stuff I download and stuff she likes to watch overlap. This is what I refer to as the “Oh Shit” zone. Problems that arise in the Oh Shit zone usually happen when someone, in this case my sister, wants to use one of my machines, and has no idea how to. I guess she experimented and figured out I had a dropdown documents folder, and somehow got into my videos, and subsequently my TV. This is where Windows Syndrome kicked in.
Windows Syndrome is what I call the computer using habits of Windows users. They like everything to be all GUI friendly and clicky. My machine is not like that. At all. I play all my movies and whatnot from CLI. I barely use a GUI file browser. It stands to reason that I was unaware that the standard action for clicking a media file was that it would open in SMPlayer, a decent GUI for mplayer, and a SHIT GUI for mplayer-uau. I use mplayer-uau. I happened to go out last night with some friends before the new semester starts, and when I returned home, I remembered that I had left my SD keyring in my machine, and that it was completely unlocked and in KDE. This is where I got a bad feeling.
On reaching my computer, I prepared for the worst. I didn’t prepare enough. What greeted me was KDE, with no window borders, dmesg shitting bricks, conky going crazy, and a completely unresponsive desktop. Ctrl+Alt+fn+F2′ing to VT2 (sorry, apple keyboard) revealed something I did not expect. A shitload of smplayer cache processes, AND A BURNT OUT CORE. How in the hell did my sister manage to burn out a core by just watching a video in a player that throws a minor segfault on good days?
This is where I got a bit scared. I looked at the hostname, and that was when shit hit the fan. This was not my desktop, this was my main rendering box. An octocore Xeon setup. A very expensive octocore Xeon setup. I didn’t pay anywhere near full price as I got it second-hand but it still cost me rather a lot. I’ve since disabled the core in /sys/devices/system/cpu/online but I’m still running some tests on my beautiful ECC registered RAM and my Quadro 4800 but I am hoping extremely hard that it was just the CPU. Needless to say she won’t pay for it.
On the side, and this is just icing on the cake, she got pasta sauce or something on my insanely sexy scope node mouse. I laughed at that. Then I remote wiped her laptop. Petty vengeance is so satisfying. That concludes this episode of Emess Hates People Who Touch His Stuff.
This One’s on Kovensky
Kovensky linked me a VERY interesting article, particularly for Sonictail I believe: http://insanecoding.blogspot.com/2009/05/will-linux-ever-be-mainstream.html
I like how he goes on about Qt being awesome because frankly it is. Then Nokia pumped this out at the end of last year and it looks to finally be going somewhere interesting: http://labs.trolltech.com/blogs/2008/12/02/widgets-enter-the-third-dimension-wolfenqt/
The stuff in the first article is quite a good read for any tech people, especially the part at the end about how fixing the way X and *nix sound servers work coupled with Qt/OpenGL/Phonon would revolutionise gaming. I for one would love to see a natively built game using Qt like that. With a bit of simple hax it could even be rigged to work on Sony’s Phyre Engine, given it already uses OpenGL. Phonon support would not be hard to add, and if Phonon got support in the PS3 firmware Qt itself wouldn’t be far behind.
I can only see coding the PS3 interface from Sony’s part becoming easier with such a toolkit, if they are not already using it secretly internally, CyberLink seem to be.
Which actually gets me a bit annoyed. Has anyone else attempted to run CyberLink PowerDVD HD in WINE? It came with my BDROM. The first thing I noticed after it failing to do anything useful was that it used Qt, which really makes me wonder why they haven’t put out a *nix version of the software. Sure it uses custom UI blobs, but they’re all included in the binary and it should run perfectly well natively. The only problem I can see is that external software CyberLink provide, such as ArcSoft’s Home Theatre software, won’t run natively.
If a company has gone to the trouble of using a toolkit that runs in *nix, why haven’t they spoken to affiliate companies about a 3rd party native DTS-MA/True-HD decoder? Plenty of soundcards support *nix, and with it the decoding of these formats, so why are we not seeing anything? I think that article above really explains this quite well. Linux already has excelletn MPEG4-AVC decoding support, with VDPAU which even outperforms DXVA as far as GPU assisted decoding goes, why don’t we have the same thing for audio?
I seem to have answered my own question, the “audio situation” in linux specifically is terrible. FreeBSD has no problem. Solaris has no problem. OSX has no problem. Linux has OSS, ALSA, aRTS, PulseAudio, ESD… I could go on. They are all slow, they are all a pain to configure. Phonon is excellent but still acts as a server of sorts. All the factioning is destroying linux.
NB: I could be incorrect about CyberLink using Qt but I don’t believe so. Point still exists that we have the option of having an excellent OS yet factioning keeps destroying that possibility.
I Strongly Dislike Open-Source Developers And So Should You
Open source projects are awesome, right? They provide usually excellent programs, with source that allows any random developer to hack it apart and improve and bugfix. They allow people like Michael Niedermeyer to yell at people and be tyranical about media players and what code is allowed (fuck you I want matroska editions, uau plz2provide) and they allow people like the great folks at the Chakra Project to make what I like to refer to as KDE on crack.
Yesterday, I went about the usual task of adding another disk to my LVM grid. I had also upgraded from KDE 4.2 to 4.3, and because KDEmod is awesome, I assumed there would be no conflict with the absolute retardation that stock KDE in Arch had introduced with its crazy package splitting (fuck you Debian for encouraging people to split shit) My LVM went fine. I had 950GB of space floating there for the ComSSA LAN this weekend. Excellent. Reboot. FUCK.
Somebody had FORGOTTEN TO TELL ME AND THEN PROCEED TO MAKE NO NOTE OF IT ANYWHERE AT ALL that due to Arch’s dumb splitting, KDEmod had been forced to rename the ~/.kde4 dir to ~/.kdemod4. I rebooted into a system that appeared to lack ALL my settings in their entirety. A simple symlink fixed this but the time it took me to find out about it was crazy. This is by no means the biggest cockup Arch has done recently, I recommend reading http://archlinux.me/brain0/2009/08/16/shit-happens-when-you-party-naked-or-use-crappy-shell-scripts/ for a good read on how to fuck up a repo, admitedly I have done this to Ophion twice, but not quite as spectacularly.
I guess I can complain about Arch but it is still my favourite distro, at least until FreeBSD implements Pacman, and then I am getting the fuck away from Linux. Arch is quirky, and it has some really random bugs that no other distro seems to have, expecially involving wine and ffmpeg, and the 64bit version is a load of shit for the most part. Curse their insanely good package manager for keeping me in Linux-land.
Now to finish all that Gulden Draak in my fridge and worry about finding a good window border style and wallpaper, because I am finally changing my desktop for the first time in 3 years. I might even ditch KDE, who knows~
y u do dis firefox?
I have been a long time hater of Firefox since about 1.0 came out. I remember loving it back in the 0.8 days, before it turned into a bloatmare (bloated nightmare, I am a master of the English language obviously) So last week, when I had so many tabs open in Opera, Arora, and Konqueror and decided I would nuke them and reopen later to see what I had up and kill the irrelevant tabs. I was then faced with a choice: Midori, Galeon, or Firefox. Now all three are GTK so should be as bad as each other, but I figured Midori was just an Arora clone (or vice versa really) and Galeon is old, so I went with Firefox. Wrong Fucking Choice.
I am wondering if anyone else has had this issue with Firefox, I believe it only effects *nix users, as an OSX G5 user I know has said it happens to her. One certain pages, but not all, and it seems to be somewhat random except GMail ALWAYS does it, cause my HDD to start grinding. It makes a terrible noise like the arm and head are ramming into the platter like a 15 year old virgin and his birthday whore in this terrible cacophany of OH GOD MY HARD DISKS FFFFFFFFFFFFF. Has this happened to anyone else?
That’s about all I have to say on the matter besides for Firefox 3 still rapes memory but isn’t too bad without any extensions, and it’s still an ugly as fuck GTK application.
All about makepkg and splitpkg .conf files
Today I was looking into ICC (Intel’s C/C++ Compiler) after someone in #x264 mentioned it. I was wondering if it was possible to not only customise a pkgbuild, but also the entire build environment as well. Obviously it’s possible if anyone takes the time to read makepkg.conf, however it got me thinking on some other things. Makepkg, for those who aren’t aware, is like a lot of the other Arch tools: simple shell script. The same goes for abs, and various other daemon and startup related things Arch does.
Customised compiler usage, or “I want to use ICC on everything”
To configure makepkg or splitpkg for ICC, simply open the required conf file, skim down to the ‘Architecture, Compile Flags’ block (for cleanliness rather than necessity,) and add in CC=icc and CXX=icpc. That’s it. As makepkg is a shellscript, it automatically loads the correct info into the build environment when running your pkgbuild. For an individual pkgbuild once off or whatever, add those variables to your build block pre-make.
Package differences and binary patching
I only noticed this feature earlier today, but makepkg actually has an xdelta module. One could add in the xdelta flag (read the source, too much :EFFORT: for me) and generate binary patches of previous patches, allowing relatively small changes to HUEG packages to not raep the connections of Indonesian, Brazillian, and Australian users. You could also stick Options=(xdelta) in your pkgbuild I guess. You’d need to have your previous package on hand however.
What the shit is splitpkg?
Chakra and KDEmod users, FLOCK! Splitpkg is the build system Chakra and KDEmod development uses. It’s an exact copy of makepkg and makepkg.conf, only the config is edited to allow for building regular packages as well as KDEmod packages. Splitpkg automatically outputs the built packages in the directory of choice, as well as a few other things.
I also run Quadpkg on my machine, another variation that compiles for -j5, outputs .tar.xz packages, and does some other handy optimisations for quadcores. My quadpkg is also configured for ICC now, which hopefully gives it more oomf.
There are other things one could try, for laziness, like distpkg, which is identical to makepkg, only setup to always use DistCC. Some things are better off not being done over distcc, but really it’s as simple as editan the pkgbuild for this, however, :EFFORT:
Now to figure out why nVidia is still broken, and then do some ICC builds of x264, ffmpeg, and mplayer. Ophion is sorely in need of repository updates on these as well as Aegisub, and quadcore optimisations could be interesting, although not too big a difference is for sure.
Remote Dependency Fetching
edogawaconan came to me today with a rather pitiful story. While Australian internet is notoriously fucked, it does maintain fairly OK speed, at an average of 1.5mbit or so. Indonesia lacks our silly bandwidth allocation but really lacks in the speed dept, to the point where downloading 35MB of makedeps or deps for a PKGBUILD takes forever. I think part of the problem is because Linux is generally faster than Windows, one tends to notice the internet being slower than the rest of the system.
So to combat this, I wrote a small bashscript that when pointed to a PKGBUILD file and given an output, it will get the URL’s for the deps and makedeps and cat them to a file. The script can easily be modified to SSH into your university and download for you, or just take the output file along with you to uni and run a wget -i output. Take the downloads deps/makedeps and either copy them to /var/cache/pacman/pkg and run something like makepkg -sri to install deps, compile and install the app, and remove makedeps.
I could of course add more functionality to the script but I don’t really care, and like always there is no help or support or whatever else I guess. Syntax and link below:
getdeps /path/to/PKGBUILD /path/to/output
I would recommend chmodding it +x and possibly sticking it in /usr/bin/ for ease of use, and of course this only works in Arch (including Chakra and Ophion,) and POSSIBLY Arch derivatives. The above example assumes it is in /usr/bin/, but if not and it’s in your current working dir for example, ./getdeps.sh in out and whatever.
EDIT: Updated with edogawaconan’s helper stuff, because I don’t care for helpful announces on failed syntax.
http://ophion.pastebin.com/f79a80758
I Hate Typesetters
I really hate it when typesetters don’t set the resolution for the script properly. Especially when I just burnt 350MB of my bandwidth on an encode and script that isn’t that great. Not to say it was bad video wise, but I saw no reason for the HUEG filesize given that on my 1080p monitor the SD encode looks about the same.
What does shit me off is when the subs are set for 704×400, and libass being broken as fuck but fairly good at what it does scales the \pos tags in the sub track, as floats. VSFilter on the other hand (I’m assuming here, but it seems the most valid explanation) uses an integer. For the non-coders, this means that when libass scales positioning tags, it creates decimals, while VSFilter truncates that down to a solid number. That or VSFilter is capable of rendering to decimals, whichever. Point is libass can’t render to decimal and therefore shits itself.
It’s really not hard for typesetters to fix this. Static and Eclipse do it all the time, Menclave are starting to. Shinsen on the other hand seem to think there is nothing wrong with it. From a windows user point of view, yeah, its fairly ok, as 99% of windows users will be using VSFilter anyway. However SHS DO tell people to use mplayer on linux/OSX/whatever, which uses libass, thus rendering incorrectly like so.
Clearly not everyone will be pissed at this, but for the increasingly large number of mplayer users, it’s goddamn annoying and so easy for the group to fix. You can argue the user can demux it themselves and fix, but honestly is it that hard for the typesetter to do? I normally get the SD encode and accidentally grabbed the HD today, I guess I learnt my lesson.
TL;DR fuck shinsen and any other typesetters that don’t set the res properly.