Advertisement

Photoshop and anti-aliasing

Started by February 24, 2003 08:06 AM
6 comments, last by Wires 21 years, 6 months ago
hey everyone, I''m looking for some help.... the game I am working on is 2d and for transparency alpha channels are probably going to be the way we end up going, BUT, in the mean time we are using one RGB colour as transparency.... I need to know how to make photoshop stop making nice blurry edges! I''ve been messing around with layer blending options and everything for ages now but can''t seem to work out what to do.... any help is greatly appreciated
If you''re going with a magic color for transparency, that color has to be absolute. Any variation in RGB values will make it not match, even if the change is minute. You could try adding a +/- buffer to catch those pixels that are slightly off, but I find that''s usually not the best way to do it. Try turning off feathering or antialiasing for the tool you use in Photoshop. I know off hand the text tool defaults to "Sharp" antialiasing, setting it to "None" fixes it. There probably is a corresponding way to do it with whatever tool you happen to use.
Advertisement
Actually, working with the antialiasing is not necessarily a bad thing. Your real problem will be what colour you antialias on.
So for instance, if you use RGB(255,0,255) as your transparency colour. You dont want the antialias to take place with that colour as you would end up with ugly pinkish edges.
What you should do is antialias on a medium grey background (or better, on whatever colour is dominant in your game background, but grey usually works fine).
Then get rid of the grey background using a non-antialias selection and put the result on a layer above your pure magenta background.
That way you get a bit of antialias which actually looks OK, while still having the colour keyed transparency.
If it doesnt make sense, tell me, I am sure I can do a little example to show you. (unless you go for a pure non-antialiased edges for your sprites).

Sancte Isidore ora pro nobis !
-----------------------------Sancte Isidore ora pro nobis !
quote: Original post by Wires
hey everyone, I''m looking for some help.... the game I am working on is 2d and for transparency alpha channels are probably going to be the way we end up going, BUT, in the mean time we are using one RGB colour as transparency.... I need to know how to make photoshop stop making nice blurry edges! I''ve been messing around with layer blending options and everything for ages now but can''t seem to work out what to do.... any help is greatly appreciated


Using photoshop to make 2D game art is a good choice but you have to make sure you do certain things.

Each tool can be set to anti-alias or not. The selection tools etc - look in the "Options" toolbox and uncheck the "anti-alias" option and set tolerance to 1. Always do this.

Painting tools are different though. The paint brush will always use a blurry edge. Never use the paint brush.
Instead, use the Pencil tool. The name is meaningless - this tool will always have hard pixel edges. Always use this to draw with.

And when using the eraser, set it to "Pencil" mode, in the options box, to make nice hard pixel edges.

Another tool you''ll find usefull is "select colour range" under "select" on the menu bar. You can use this to make sure all the areas you think are the transparent colour actually are the transparent colour. With the fuzzyness set to 1, any colour that doesn''t match the selected colour exactly will not be selected when you press OK.

For local multiplayer retro themed games, visit Domarius Games.

Nice one... thanks alot people!
you can use whatever tools you want, just make you outline and fill shapes with the pencil tool, then set the layer properties to "protect transparency" - that way you can smear all you want without blurring the edges.

I´d try and do without antialiasing at all, even if your background matches the intended game bg well you can still get some unwanted border effects... I usually work on a background close to what´s going to be in-game (merely for getting the color balances right), but leave the edges hard with the technique above - that way your sprite can be transferred to other bacgrounds as well without getting ugly borders.
Advertisement
Yeah I forgot to mention. Don''t rely completley on the things I just said in my last post. Just use a combination of those ideas WITH your normal photoshop stuff, to ensure your sprites have a hard border, and a consistant "transparency" colour.

For local multiplayer retro themed games, visit Domarius Games.

Another thing that will help you control this is to work with the alpha channels in photoshop. Many times you are going to get some ugly anti aliasing problems when you flatten an image. So, lets say you''ve made a beautiful composition in Photoshop... and you''ve got.. oh a good 5 or 6 layers in use with transparencies defined by each layer, and these are all on top of a solid background layer thats... lets say, is black.

Well, common sense says to flatten the image and then use the paint bucket tool to fill in with your transparency color. THIS is what is causing an ugly anti-aliasing.

My suggestion is to hide your background layer, "merge visible", selection - load selection - merged layer transparency, save the transparency, then look at your newly formed alpha channel. If in the alpha channel you have anything that isn''t either solid black or solid white, you''ve got a problem. Grays cause an anti-aliasing nightmare.

Its too bad that you can''t actually USE the alpha channel rather than the "magic color" method - because the work around for this is simple. you just bleed out the color on all your edges and let the alpha channel then blend it with its new background when put in the engine...

Since you can''t do that, you CAN use it as a reference. First and foremost, use brightness/contrast on your alpha channel to force it to be black and white solidly. Then even your alpha channel out so your lines aren''t too jagged and ugly (but still solid black and white). Once you are done with that, click selection, load selection, Alpha 1, copy, paste that into a new document, and you wont'' have any more anti-aliasing woes

Or, if you want a quick fix, get the image down to your foreground and a background (two layers), and adjust the levels on your foreground to make the blackest black not so black any more Then when you flatten it and fill in the background with your magic color (fill tool at 0 tolerance), it should be very consistent with not tearing apart the contour of your image.

Hope that helps
- T. Wade Murphy

This topic is closed to new replies.

Advertisement