What would you do if your clergyman said you were no longer worthy of your family and they were taken away from you?

To most of us, it's preposterous to even imagine such a thing happening. But I spent some time this week with a 22-year-old Idaho man who claims that's exactly what happened to him.

Wendell Musser is desperate. Vivian, his 21-year-old wife, and Levi, his 1-year-old son, dissapeared, he says, and he blames polygamist sect leader Warren Jeffs and the Fundamentalist Church of Jesus Christ of Latter Day Saints (FLDS).

Musser tells me Jeffs, whom followers believe is a prophet, has about 180 wives. He said Jeffs asked him to live with and protect eight or nine of them in a secret hideaway while Jeffs was running from the law.

Ultimately, Jeffs landed on the FBI Ten Most Wanted list on allegations of arranging marriages of young girls to adult men. He has since been caught and is awaiting trial.

But while Jeffs was on the run, Wendell Musser and his wife Vivian took care of Jeffs' wives. Musser told us Jeffs would occasionally visit. He said the "prophet's" spouses didn't know quite how to act when their husband came a callin' because with so many wives, they spent time with him infrequently.

Last June, Musser was pulled over by police for DWI. He spent two days in jail. When he got back to the secret hideaway in the remote Colorado mountain town of Westcliffe, not only were Jeffs' wives gone, but so were Vivian and Levi.

The word from the church: Musser had let the prophet down, and as punishment his family was being reassigned to another man. Musser has no idea where his wife and child are.

Musser's marriage to Vivian was an arranged one, but he said both of them did fall in love with each other. He wants his wife back, and is heartsick without his baby. He has written letters to Jeffs in jail asking for the wherabouts of his family, but has received no response.

So he has now filed a most unusual lawsuit, which asks a judge to order Jeffs to talk. If he wins the suit and Jeffs doesn't open up, the penalty could be prison time, which won't do much for Musser since Jeffs is already being held in jail without bond. There has been no comment to us from Jeffs' FLDS church.

Musser, who is now a former member of the flock, hopes the "prophet" (who also happens to be his uncle) has a heart. And for that matter, Musser also hopes his father and father-in-law have hearts because he believes one or both of them may also know where Vivian is.

However, the two of them, along with most of Musser's 45 sisters and brothers remain in the church, and they all do their best to shun Wendell Musser.

-- By Gary Tuchman, CNN Correspondent