I use Handbrake to convert a lot of my DVDs to store on my teradrive then use on my PS3. They used to have a PS3 preset, but not in the newer versions. Fortunately for YOU, I still have the preset, and use it regularly, successfully.
Use either MP4 or M4V format (M4V is only useful if you're going to title each chapter, a function PS3 doesn't care about, but you can access on other platforms).
Video Codec: H.264 (x264)
Framerate: Same as Source
Average bitrate: 2500 kbps
Strict anamorphic video output
Audo Codec: AAC (faac)
Mixdown: defaults for track type
Advanced settings-
Motion Estimated Method - Uneven Multi-Hexagon
All other advanced settings - Default
CABAC Entropy Coding - Checked
Remember to check and see if you need to deinterlace. If, when playing the DVD on VLC media player, you see lines across the screen, then you need to have deinterlace enabled.