There are many good reasons why we do not commonly grow the more unusual fruits and vegetables. But on the other hand there are some very good reasons why we should.
The most likely cause for any edible plant being ignored is that rather often they are just not very good to eat. The Strawberry tree, Arbutus unedo, is a miserable fruit, indeed the Latin name translates as "eat one only". Scurvy grass, Cochlearia officinalis, may be good for you but I'd only eat it again if faced with scurvy! The fruiting goosefoot; Beetberry, Chenopodium capitatum, has small pasty, seedy, ungrateful strawberry like fruits that even the birds do not like. Likewise the blue passion flower's orange fruits have an edible pulp but it's so scanty and poorly flavoured it's hardly worth the effort. Yet each of these could soon be improved with breeding to make far better fare.
Many potential crop plants are only edible in the leaf; described so frequently as "may be eaten like spinach" or "may be boiled as greens" such as the Turkish rocket, Bunias orientalis. or the ice plant, Mesembryanthemum crystallinum/ Carpobrotus edulis . This last is the most drought resistant crop I've ever grown but is poor eating unless desperate. Of course we each enjoy eating some spinaches in moderation. However plants that only produce a bulk of some fibrous leaf rarely find a ready place in our gardens. Some such as Good King Henry, Chenopodium bonus-henricus, are useful though, as they come early in the year when other greens are scarce. (I find stinging nettles are a good alternative to most other spinaches, and, like dandelion leaves are very healthy for us, but few ever want to give either valuable space.)
A big handicap is poor keeping qualities. Some fruits are notorious for going mouldy once picked, Raspberries for example, but these are tasty enough to still win a place in the garden. Other less tasty berries such as Salmonberries and Dewberries that mould as easily just do not get a look in. Roots which do not travel or store well are little use commercially and so often remain unknown to the average gardener. Who knows such as Skirret, Sium sisarum, and Earth chestnuts, Bunium bulbocastanum ?. Even the better known Salsify and Scorzonera which bleed once harvested are not sufficiently different to carrots or parsnips to make them really popular though actually they're very tasty.
Being prone to pests or diseases, or just downright difficult to grow or crop is a serious drawback. Hamburg parsley, Skirret and many other umbelliferous plants suffer carrot root fly attacks but do not yield anywhere near as well as carrots. Some edible crops such as Ysaño the tuberous nasturtium, Tropaeolum tuberosum, Oca, Oxalis tuberosa, and Tomatillos, Physalis ixocarpa, need greenhouse culture and cannot compete with more popular alternatives for the limited room. (Especially as they're not that good to eat either, feel free to like them if you do though)
Many of the edible berries such as the Cherry prinsepia, Prinsepia chinensis, Worcester-berries, Ribes divaricatum, and Barberries, Berberis vulgaris, are simply thorny brutes that do not sit comfortably within the modern small garden, though they make good barriers. (Anyone wishing to grow Worcesterberries would do well to take out insurance before picking their crop; they are one of the most difficult of all as they have extremely vicious thorns.)
Sometimes we associate plants so strongly with their flowers or foliage that we find it hard to see them as food. Dahlias were originally introduced for their edible roots and not for their flowers! Likewise Chokeberries, Aronia melanocarpa, are grown for their brilliant autumn colour and not for their edible fruits which make a good jam. Fuchsias like Roses have an edible berry, but either is not that tasty and similarly all breeding for both has been concentrated on the flowers so good cropping sorts have never been developed.
Very much on the plus side is that many of the least usual crops are so rare they have few common pest or diseases (other than birds, slugs etc.) If you grow Alliums and Brassicas then there are a host of common problems they always risk attack from. If you grow Asparagus peas, Lotus tetragonolobus, they are unlikely to be attacked as there are probably few others of the same growing for miles around for the problems to come from. Some alternatives such as Kohl rabi are closely related to the commoner brassicas but are inherently tougher and less attractive to pests.
As many of the unusual vegetables are from different families to our everyday crop plants then they not only suffer from fewer of their associated ills but they also make differing demands on the soil. This means that they are easy and beneficial to include in our rotations extending the time before the more susceptible crops return. Likewise the unusual perennial crops enlarge the ecosystem in the garden providing new pollen and nectar sources, differing leaf and petal falls and extracting and returning a variety of different elements with the soil.
The greatest advantage coming from growing unusual foods though is the nutritional aspect. It is beyond dispute that the more wide ranging the diet the more likely one is to get all the trace elements and vitamins necessary for a healthy life. The modern junk food / instant meal 'diet' is based on very few plants, probably less than a few dozen in total, whereas our ancestors ate several hundred different plants. Admittedly they lived predominantly on beer and bread but they ate foods gathered from the wild in profusion in season and again ate them as medicines when 'run down'. Indeed in spring they made great efforts to imbibe tonics consisting of the expressed juices of dozens of different herbs. Even today some peoples still eat a wide variety and I've heard it said that the Japanese strive to eat thirty different things a day and a hundred and fifty each week.
So which of the unusual crops are worth the effort? Well over the years I have grown and tried almost every edible plant I can lay my hands on. By diligently searching the seed catalogues and nurserymen's lists I have managed a surprising number. As I said earlier on in this article the majority are somewhat disappointing from the culinary point of view. For example I find it hard to recommend Medlars as I do not like the taste of the 'bletted' (half rotted) fruits but others disagree and as the small trees are visually appealing they do deserve space in a large garden. Nonetheless some crops have been to my taste and so I can wholeheartedly recommend you try the following some of which you may already have or know.
Rowan trees are compact and their fruits make delicious preserves and a large fruited sort, Sorbus aucuparia edulis , is worth getting. Hazel, cob and filbert trees tend to get too big for small gardens, they also do better if a couple of sorts are planted but the nuts are very valuable protein sources and delicious, I strongly recommend the Red-skinned filbert which I find makes a more compact bush than the others.
The berries are more use to most gardeners as they take less space and can be pruned and trained more easily than trees. The best of all the rarities is the Japanese wineberry, Rubus phoenicolasius, which is like a small red blackberry -the birds do not bother them, they are very attractive in leaf and stem, the fruits are delicious and the stems are not so much thorny but bristly. The Josta is a hybrid halfway between a black currant and a gooseberry, thornless, high yielding, easy to grow but gets bigger than either parent. (Do not even try the Worcesterberry which is similar but as mentioned above too thorny for use!) The Chokeberry, Aronia melanocarpa Viking, is worth growing as it's berries resemble black currants and have very high vitamin C levels but need making into preserves as they are unswallowable fresh! Likewise the berries of Oregon grapes, Mahonia aquifolium , make a good preserve and it's worth growing just for that purpose though of course it's also a well known ornamental.
Amongst the annuals one that ought to be in every garden is the Ground cherry, Physalis pruinosa,/pubescens this close relation of the Cape gooseberry is much hardier, more compact and will grow happily outdoors. The fruits taste vaguely like pineapple and it is a very productive plant. An annual with an edible pod is the Asparagus pea, Lotus tetragonolobus, this is quite tasty when small, it will grow in poor soils and the seed can even be roasted to make a caffeine free coffee substitute.
As mentioned above many of the rarer root crops are not much competition for our favourites. Though very well known by name Jerusalem artichokes are not grown in the majority of gardens, they are one of the easiest and most reliably productive foods yet few appreciate their taste, texture or wind production and sadly they only store well left in the ground. Similar but much smaller and lower growing is the Chinese artichoke, Stachys affinis, which I quite like but it is even more fiddly to peel than the Jerusalem. Horseradish is another well known but rarely grown root, it is possibly very healthy eating and is also a very good companion to potatoes so it should be in everyone's garden, and once in you'll always have it! Another must is Rocambole which is a variety of garlic but very decorative with coiled stems and a milder taste.
Thus although there is little likelihood of most of these more unusual crops ever becoming widely popular there is a lot of sense in trying them out. Of course if you are very limited in space then they are probably not as good a value as strawberries, carrots and cabbages. But if you can try the odd one every now and again you may find that some are really to your liking and worth growing in some small proportion every year. And of course there is the immense fun of growing something different anyway.