"And thou shalt remember all the way which the Lord thy God led thee these forty years in the wilderness, to humble thee, and to prove thee, to know what was in thine heart, whether thou wouldest keep his commandment or no." (Deut. 8:2)
This is one of life's great mysteries. My husband is ministering almost daily to a neighbor he led to a saving knowledge of Jesus Christ, but who struggles with constant pain and deteriorating health. "Why can't I just go to heaven and be relieved of all the pain?" he wonders. Philosophers have grappled with this down through history, and hundreds of books have been written trying to find meaning to what seems to have no meaning at all.
God does not hesitate to point out the reality of suffering. And to those who are Spiritual enough to grasp it, He offers insight into the lofty purposes behind the pain. For instance, in this chapter in Deuteronomy, Moses reminds the nation of Israel, it was God who led them those forty years in the wilderness to suffer hardship, deprivation, and chastening. And then he tells them some things it had done for them...and, by extension, what it does for us.
First, he says, suffering humbles us. There's nothing like walking through a dry, barren wilderness to take the strut out of your step! That's why the purveyors of the so-called "health and wealth" gospel (not to be confused with the Gospel of Jesus Christ) are not known for their humility. But humility, according to James 4:10 is a boon to our Christian experience, and is the first step toward elevation by God.
Second, testing brings to the surface what we are really made of ("to prove thee"). When we are hard-pressed, the best—or worst—part of us is seen. It reveals what's in our hearts every time. Words of praise to the Lord that come so quickly to our lips during times of prosperity and ease, can seem to stick in our throats because of bitterness in our hearts over the perceived unfairness of God.
Third, in verse sixteen of the chapter, Moses goes on to tell these tired, weary travelers that God led them through "that great and terrible wilderness," with its serpents, scorpions, and drought, not only to humble and prove them, but "to do [them] good at [their] latter end." You see, there's a purpose in the pruning: fruitfulness (John 15:2); and there's a scheme in he suffering: freedom (1 Peter 4:1b). Just as suffering can bring new life into the world, it can breathe new life into our Christian experience. As the verses in John and 1 Peter indicate, it makes us more useful to God and less useful to this sinful world.
These few thoughts of mine may not be the whole answer, but they're an important part of it, I think. And if they bring a measure of peace to your weary, suffering soul today, I am blessed.
"Who comforteth us in all our tribulation, that we may be able to comfort them which are in any trouble, by the comfort wherewith we ourselves are comforted of God." (2 Cor. 1:4)