Features as elements of association

Associations are usually not made on raw data; instead, certain features of the raw data are identified, and the mechanism of association then recalls some other data that has one or more similar features. In other words, the notion of 'similarity' applies to features of the data sets, i.e. the "data type" used when evaluating similarities is 'feature' and not 'raw data'.
What is the nature of the raw data involved in associations, what are the inbuilt feature extraction mechanisms, and whether or not feature extraction mechanisms can be learned, are all essential to understanding the mechanism of association.


Static-data associations
A very common type of association is the one made on static data, e.g. still images or invariant sounds. The defining characteristic of static-data associations is that the element of time is irrelevant in performing the associations.

  • example of static-data associations on images (a real-life example that happened to me)
    i was listening to a radio show that played a psychedelic tune, and at the end of the tune the d.j. made a comment about that tune allegedly suggesting a colorful image with randomly placed and randomly colored bubbles. I was paying attention at what the d.j. was saying, and as he was describing the allegedly suggested image, a mental image was incrementally "maturing" in my mind following his description. I then closed my eyes in order to better visualize this image, and, after briefly having an inverse-video representation of my mental image, i instantly had a recollection of a another similar image consisting of a letter written out of colored bubbles among other color bubbles. Although the image recollection was instantaneous, it came without any context information, i.e. i did not automatically remember where and when i have seen that image before; instead, i had to consciously focus on finding the context of that image, and only then did i realize that it was part of a vision test that i was submitted to when enrolled in the army (i will mention here that the time between this recollection and the army vision test was about eight months).

    In this example the elements of association were exclusively image features, namely the bubble pattern in the image and the colorfulness of the bubbles, and the association was performed on purely static data (i.e. the trigger image and the recollected image) with no time element being involved.
  • example of static-data associations on sounds
    Simply hearing a certain sound with a certain timbre may remind us of a tune that uses (e.g. begins with) that sound, or listening to a voice recorded on a tape will enable us identify the person who's voice is recored. The features of the time-invariant sound itself are the exclusive elements of association, i.e. the associations are performed on purely static data (i.e. the non-fluctuating sound) with no time element being involved.

Dynamic-data associations
Associations can also be made on features of dynamic data, i.e. the very dynamics of a data sequence can be associated with a similar dynamics of another data sequence. As is the case with static data, the notion of 'similar' applies to features of the dynamic data (both the triggering data, and the recollected data), but in this case the features will include the element of time.
  • example of dynamic-data association on sound
    it is enough to hear a certain tempo of a tune (e.g. the starting tempo) to be able to say that the tune resembles another tune.

  • a more complex example (involves trans-modal association, details below)
    let's imagine we watch an old black-and-white movie in stereotypical fast motion, and then, later that day, we're searching for a tune on an audio tape by pressing the 'cue' button on the tape recorder: when we will hear the fast-forwarded sound on the tape we will (probably) immediately recollect us seeing the fast-motion movie. In this example, the dynamics-related common feature is 'fast' (or maybe even 'fast forward') and it is this feature that is being used in the association, i.e. the association is based on a feature extracted from the dynamics of the data.
    • note: in this example, the end-to-end association process also involves the concept of "fast" (or possibly "fast forward").

Trans-modal associations
Another important remark that can be drawn from the example above is that the type of data involved in triggering an association can be different from the one type of data that is returned by the mechanism of association: in the example above, the triggering data type was auditory, while the recollected type was an entire scene of us watching the movie (this scene would consist of a fuzzy mental image, possibly accompanied by some sensations related to how comfortable we felt in the chair where we were sitting while watching the movie, etc). While the element that triggered the association was a feature of an auditory signal, the recollected data type is much more complex (it's a scene) and the recollected data feature was purely visual.

In fact, the feature extraction mechanism can go much further then it was illustrated by the examples above: specifically, just about any sensation that we have, or even mental states that we are not aware of (e.g. the "presence" of a concept in our mind), can be elements of associations.
  • example 1:
    let's imagine that we have a rolled-back video tape and we want to position it to a certain scene (e.g. we want to show that scene to a friend): What we will do is to take the tape, then approach the VCR, insert the video tape in the VCR, and finally we'll cue the tape in search for the scene we're looking for. Now let's imagine that, later that day, a friend calls and asks us for a certain audio tune, and let's assume we do have that tune recored somewhere on an audio tape. We'll answer to our friend that we have it she can come and take it. What we shall do next is to go to our tape rack, pick up the tape, insert it in the tape recorder, and then cue in order to find the desired tune. Let us now consider two scenarios:
    • if we were paying enough attention to the intentionality of finding a specific position on a tape in both situations, then it is very likely that the moment we'll start being aware that we are about to search for the position of a tune on the audio tape, that very moment we will have a flash-back with a recollection of us searching for the video scene on the video tape earlier that day. In this case, the very intention to find a position on a tape will be the element of association.
    • if, on the other hand, we were not paying enough attention to the intentionality of searching for a position on a tape either when searching the video tape, or when searching the audio tape, or both, then we might still get a flash-back the very moment we press the cue button on the audio tape recorder and start witnessing the process of a machine cueing a tape. In this case the, the very scene of a tape machine cueing to a position on the tape will be the element of association
  • example 2:
    Scene 1: I have to take the subway from station 'Lorem Ipsum'. As i climb down the stairs, i see a lit panel with the name of the station. Unfortunately, i do not remember if i actually read the writing on the panel or not.
    Scene 2: I am sitting at my desk writing a paper. On my left there is a bottle of wine wrapped in paper. At the left of the bottle there's a book that i use as reference for my work. While writing, i need to check for something in the book, so i turn my head, read what i have to read (didn't have to turn pages), and then i re-direct my eyes back to the paper that i'm writing (which is in front of me). Instantly, i have a recollection of the lit panel with the 'Lorem Ipsum' written on it, but i'm not yet aware of the surroundings of my mental image; it will take a conscious action to mentally "look around" the panel and see the entrance of the 'Lorem Ipsum' subways station.

    What actually happened in this case was that, while i briefly turned my head towards the book on my desk, my eyes must have briefly scanned the label on the paper in which the bottle of wine was wrapped, but i am absolutely sure that i did not consciously read the label. The label contained the name of the store from where the bottle was bought (it was not me who bought it and i had no idea where it has been bought from), and the address of the store was 'Lorem Ipsum 123', etc. Apart form the name 'Lorem Ipsum' itself, there was no resemblance whatsoever between the shape or "style" of the writing on the gift paper and the one on the subway station panel, there was also no resemblance in the color or shape of the individual letters themselves, etc; actually, all the optical features were quite different.
    There are two important remarks in this example:
    • first, the association involved a synthetic construct related to language, i.e. the name 'Lorem Ipsum', without involving any sensations (i neither heard the name of the station being "pronounced" in my mind, nor have i consciously seen - let alone read - the label, etc). Because of the inability to sense the triggering element of association (no sensation was involved), the association was made completely outside the realm of my awareness. (when this episode happened i started to analyze what caused it, and it actually took me quite some time until i've seen, and this time consciously read, the label on the bottle, and only then did i understand what had actually happened)
    • second, while this association was triggered by a synthetic data that has no commonalities with an image, i.e. the name 'Lorem Ipsum', the association retrieved a pure image. As mentioned before, i was completely oblivious to any context information when the subway station panel's image popped up in my mind, and i had to mentally "look around the panel" to start filling in the context from memory (i.e. to realize that what i actually remember is an image seen a few days ago when entering the subway station). In other words, this example is also an illustration for the fact that the type of data that triggers an association and the data retrieved by the association need not be the same.

No comments:

All content on this website is covered by the copyright policy of the AI Project.