AlisaMom, I've been vacationing in Southern California since childhood and we spend the greater part of the afternoon on the beach and in the water. While it's possible to swim in the Pacific, we don't often see swimmers out past where the waves crash. The kids in lifeguard camp swim from one pier to the other in Newport, but it's tough to get out past where the waves crash if you're not experienced. My dad used to go out into the water pretty far, starting as a teenager and ending in his 30s after he was injured by a crashing wave. It's more for jumping waves and boogie boarding, which is great fun for all ages! There is swimming involved, but not like traditional lap swimming.
Our favorite beach is Newport Beach - just near Huntington. Newport is a peninsula, and a large part of it is just about 2-3 blocks across. The ocean is on one side of the peninsula and the bay is on the other side, making it perfect for wave jumping and boogie boarding on one side and swimming on the other.
P.S. It's cold! That used to be fine with me, but it's been cold enough lately that I finally got a short wetsuit. But water temperature is not a problem for our three boys!