Although most of the food crops we grow are available as seed or plants there are some rare and unusual ones that are not easily findable. Some are also very difficult to start from commercially supplied seed as their viability is rapidly reduced as they age; and very little seed sold is less than a year old at least. (Even seed grown in the opposite latitudes ripening at the same time as our spring is only going to be transported, packaged and sold to us the following winter!) Some plants are also difficult if not impossible from seed; specific cultivars, especially of perennials, must usually be propagated vegetatively.
There is a certain pleasure starting off a plant from seed or slips that you have obtained from a fruit, vegetable or plant you have just eaten and found exceptionally good. Although probably illegal, many people do bring back seed from something they ate on holiday. Naturally there is a danger of also introducing a new pest or disease to your garden by using any 'unusual' source, but for the rarer plant species this is pretty unlikely. None the less please avoid bringing back contraband orgrowing from such as ware (the term for 'grown for eating') potatoes and so on already widely grown as crops here.
However how do you know whether or not you can grow the plant, and more importantly can you get it to crop? Over rather more years than I care to admit I have tried to crop almost every food crop in the world that I can find material for. I have been amazed at some successes, and of course there were failures, though surprisingly very few. So to start off with the bad news first; few of us would be foolish to try and grow a tea plant from a teabag, however some apparently easier subjects are just as futile. (Mind you don't let me stop you trying, just be warned that success is not very likely in such cases.)
As difficult as it gets are using seeds or plant products that have been treated to improve their taste, storage or preservation for culinary purposes. Thus as impossible as it is to raise a salted peanut, tea from a tea leaf, or coffee from a roasted bean other apparently alive seeds and plant material are to all intents dead for our purposes. For example, ginger bought as the fresh root as sold in supermarkets can usually be got to grow but the dried ginger roots that are also sold are not only desiccated but also scraped and coated with lime making them totally unviable. The similar looking fresh Galangal has however been boiled to help it keep and so will often not grow. Equally I've found it futile to try and grow green, black or white pepper; the corns have been so treated that they are effectively dead especially the white which are retted and decorticated to boot.
Likewise nutmegs are invariably well past any hope of germination, they are difficult when fresh but by the time we get them the processing and delay has made them useless, though of course quite good enough for culinary purposes. Coconuts are another waste of time; the coconut naturally germinates and grows it's first roots initially inside the husk -which was removed before it was shipped. The 'naked' coconut we buy might just do, but I have never succeeded, and bringing back from abroad a whole unhusked one would be illegal. (Also many coconuts sold are cracked and / or starting to rot -never risk eating a coconut that is dubious as it is risky healthwise; examine the eyes as rots usually commence there.)
Indeed oily seeds, especially nuts, are amongst the hardest to succeed with as they tend to go off and go rancid very quickly. Most nuts that have been shelled will never germinate, indeed in many cases the shell's attributes are necessary for germination to proceed by controlling water and air exchange with the seed within. Really fresh nuts in their shells may germinate and will produce plants, even cropping if the right conditions can be given them. However they are not necessarily going to come true and although you may one day get a nut it may not be as good as the original which may have been from a selected clone. e.g. walnuts in their shells will germinate fairly easily (if fresh) though those imported at Christmas time I have found unreliable, chestnuts and hazels equally so. But the trees produced will be genetically variable and not likely as good as a bought in sapling. Worse, it will take you many years to discover so!
For the same reason although almost any seed found in almost every fresh fruit is likely to be able to grow it may be pointless as the results will be random and barely worth the effort and the waiting involved. Though this depends on the degree of breeding and selection that has been applied to the original. Thus a wild sloe stone will invariably give a similar fruit but cultivated plum stones or apple pips will eventually throw fruits that have almost no chance of being like their parent. Some tropical fruits are relatively unbred so with these you may have a degree of success greater than with our temperate fruits which are generally more highly developed. Tamarinds, Loquats, carambolas, custard apples, mangosteens, rambutans and lychees may all be fairly easily germinated and grown on, and although their chances of actual cropping are pretty remote if they did most would probably come fairly true. (A mangosteen was fruited in a stove house in the mid nineteenth century!)
Passionfruits, tree tomatoes, guavas, kiwis, strawberry guavas and custard apples have all grown, cropped and come true enough for me. Papaws are easy to start, but not easy to crop, when they do the result is variable but still useful. Avocados, mangos and dates are very easy to start, neither come true nor are possible to crop. (Please prove me wrong) Olives are variable yet actually often grown from seed however imported green ones are unripe, the black are ripe but almost fermented to extinction in their processing.
One of the more interesting possibilities is citrus seeds, these are easy to start and grow if slow to bear but they are very unusual in that occasionally two 'seedlings' or more come from one seed! One is the true sexually produced (random) offspring but the other(s) are clones and will be just like the parent. (Mangoes are similarly polyembryonic and make interesting plants though most unlikely ever to crop except in a Stovehouse.) Peaches and apricots can be very similar to their parents so if you wish to try these then use stones from locally grown varieties not imported ones from Southern France or Italy (unless you live there that is).
Seeds that we buy to eat, such as mustard, caraway, poppy, coriander, anise, peanut, and haricot, lima, butter and other beans and so on are more of a good bet. Firstly you often get so many for so little money that even if germination is only one per cent then some plants will still be produced if the right conditions are given them. Be warned though, although I believe it is illegal (in the UK) for culinary products to be irradiated I gather that some imported seeds, especially spices, are exactly so treated, indeed even the 'normal' processing, fumigation, packaging and storing and may affect the outcome detrimentally. There is also the possibility that such seed may be of F1 varieties and so will not come true, or even worse it may be genetically modified. (To add insult to injury, literally, you might even, theoretically, open yourself to litigation from unprincipled and ruthless corporations!)
The problems of coming true are avoided where vegetative reproduction is possible. Never even consider growing banana seed sold commercially -it comes true and produces very poor seed filled bananas! The only way to grow a good banana is to get an offset of the best variety; the dwarf Chinese or Cavendishii sort which if kept warm fruits easily in a greenhouse tall enough for it (9+ft/3m.). (Plants of this, usually unlabelled as such, are often sold as houseplants in supermarkets, a good indicator of this right variety is pale purple markings in the middle of the younger leaves.) Likewise ginger, galangal and lemon grass are all easy to grow from the fresh product available in supermarkets. Sugar cane is easy but like the bananas it needs space. Pineapples are not at all difficult if they can be kept warm and light enough; the crowns once detached will root easily and crop within three years.
Sweet potatoes, yams, Eddoes and many such imported root crops are remarkably easy to grow and crop from the supermarket produce given the right conditions (warmth usually). And indeed many temperate root vegetables that have no significant pests or diseases such as Jerusalem and Chinese artichokes are much better quality and value when purchased for eating rather than from seedsmen! In a similar manner it is as easy and quicker to root fresh culinary herbs such as mints, rosemary, thyme and oregano than it is to grow them from seed. Watercress is far far better when grown from the fresh produce than from the seed!
And for those of you curious about growing on the odd vegetable or soft fruit seedlings we find spontaneously growing in our gardens: I've found asparagus, blackcurrants, redcurrants, raspberries, Japanese wineberries and alpine strawberries are all worthwhile keeping. Blackberry hybrids less so, grapes are totally pointless though easy, and seedling strawberries seem to be happier continually runnering than ever fruiting.
So, the best of luck, and if you find anything does different to the above please let me know. Now all I have to do is avoid the hitmen from the seed companies......