When he was 13, Pat's widowed mother moved to San Francisco, leaving him at the Honolulu Military Academy.