ketan agrawal

compositionality
Last modified on May 19, 2022

You have a set of reusable pieces that can be combined to form different, interesting things.

Decomposition of programs into modular, reusable unit is one example of this.

Static, Dynamic, and Requirements Models for Systems Partition (as an example of decomposition in programming.)

Some methodical activities, such as making breakfast, decompose well into modular activities as well:

tree of the decomposition of making breakfast into "make toast," "make tea," etc.

A machine learning example of compositionality

is that a computer vision model’s early layers will implement an edge detector, curve detector, etc., which are modular pieces that are then combined in various ways by higher-level layers to form dog detectors, car detectors, etc.

progression from low-level to high-level features in a CNN.

But ultimately the ways in which machine learning models, at the moment, can “intelligently” combine different pieces and parts is limited. See the tweet about DALL-E trying to stack blocks, bless its heart:

One step towards compositionality is the idea of neuro-symbolic models. Combining logic-based systems with connectionist methods.

compositionality in the human mind

how do our brains combine simple concepts into more complex ones? Even though the input we receive is noisy and finite, we can (in theory) produce an unbounded number of sentences. r/BrandNewSentence is an entertaining example of this.

Related: [1]

Bibliography

[1]
B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, “Building Machines That Learn and Think Like People,” Apr. 01, 2016. Accessed: Apr. 28, 2022. [Online]. Available: http://arxiv.org/abs/1604.00289

Links to “compositionality”

Counterfactual Generative Networks

Neural networks like to “cheat” by using simple correlations that fail to generalize. E.g., image classifiers can learn spurious correlations with texture in the background, rather than the actual object’s shape; a classifier might learn that “green grass background” => “cow classification.”

This work decomposes the image generation process into three independent causal mechanisms – shape, texture, and background. Thus, one can generate “counterfactual images” to improve OOD robustness, e.g. by placing a cow on a swimming pool background. Related: generative models counterfactuals

modularity

I think it can be really good. Fairly related to compositionality. Will have to hash out what I think the difference is there.

♏️ emacs gets a lot of its power from its modularity. Packages are just little pieces of code– you can add/remove them into your ecosystem as you please. As opposed to proprietary software, where you may be much more limited in the customization that you can do…one piece of functionality for Evernote might not carry over to VSCode might not carry over to your web browser. etc.

^it’s beautiful chaos sometimes, though! Modules designed by tens of thousands of independent humans are bound to be overlapping, redundant, and sometimes confusing to “glue together”…

Despite reading explanations and guides occasionally, I’ve never understood how Org, BibTeX, org-ref, ebib, etc. all work together. I once tried to set them up using BibTeX references in a file, but I wasn’t able to figure out how to get citations working. I ran out of time to fiddle with it and had to do all the citations manually. :(

Well… They do not. It’s up to the end users glue them together. Bib(La)TeX only take care of record books&c references in a textual flat-file database, org-ref is mostly a helper to add things to that text db and quickly link a record in an org documents, helm-bibtex or ivy-bibtex are the search-based UI to operate with.

But luckily emacs has a great community that has helped smooth out the rough edges between packages, and even make them work together to make emacs a better place.

But….is the brain modular? I think it might not quite work that way.

Towards Causal Representation Learning

Yoshua Bengio talk. Also, the associated paper.

causal representation learning: the discovery of high-level causal variables from low-level observations.

In practice, i.i.d. is a bad assumption. Things don’t stay the same distribution as train. Current DL systems are brittle.

But…what assumption can we replace it with, then?

how does the brain break knowledge apart into “pieces” that can be reused? => compositionality (thinking decomposition into helper functions in programming.) Examples of compositionality include

is it always optimal to compartmentalize / solve one problem at a time? Are there times when we might want to solve multiple problems simultaneously? (is it always optimal to compartmentalize / solve one problem at a time? Are there times when we might want to solve multiple problems simultaneously?)

thinking about this in relation to dynamic programming – I think it serves as a good metaphor for why it can be useful to think about the structure of the problems you’re solving, and if there’s any overlap / reuse that can occur….seems that this is also related to decomposition / modularity

Links to “decomposition”

Counterfactual Generative Networks

Neural networks like to “cheat” by using simple correlations that fail to generalize. E.g., image classifiers can learn spurious correlations with texture in the background, rather than the actual object’s shape; a classifier might learn that “green grass background” => “cow classification.”

This work decomposes the image generation process into three independent causal mechanisms – shape, texture, and background. Thus, one can generate “counterfactual images” to improve OOD robustness, e.g. by placing a cow on a swimming pool background. Related: generative models counterfactuals

modularity

I think it can be really good. Fairly related to compositionality. Will have to hash out what I think the difference is there.

♏️ emacs gets a lot of its power from its modularity. Packages are just little pieces of code– you can add/remove them into your ecosystem as you please. As opposed to proprietary software, where you may be much more limited in the customization that you can do…one piece of functionality for Evernote might not carry over to VSCode might not carry over to your web browser. etc.

^it’s beautiful chaos sometimes, though! Modules designed by tens of thousands of independent humans are bound to be overlapping, redundant, and sometimes confusing to “glue together”…

Despite reading explanations and guides occasionally, I’ve never understood how Org, BibTeX, org-ref, ebib, etc. all work together. I once tried to set them up using BibTeX references in a file, but I wasn’t able to figure out how to get citations working. I ran out of time to fiddle with it and had to do all the citations manually. :(

Well… They do not. It’s up to the end users glue them together. Bib(La)TeX only take care of record books&c references in a textual flat-file database, org-ref is mostly a helper to add things to that text db and quickly link a record in an org documents, helm-bibtex or ivy-bibtex are the search-based UI to operate with.

But luckily emacs has a great community that has helped smooth out the rough edges between packages, and even make them work together to make emacs a better place.

But….is the brain modular? I think it might not quite work that way.

Towards Causal Representation Learning

Yoshua Bengio talk. Also, the associated paper.

causal representation learning: the discovery of high-level causal variables from low-level observations.

In practice, i.i.d. is a bad assumption. Things don’t stay the same distribution as train. Current DL systems are brittle.

But…what assumption can we replace it with, then?

how does the brain break knowledge apart into “pieces” that can be reused? => compositionality (thinking decomposition into helper functions in programming.) Examples of compositionality include

is it always optimal to compartmentalize / solve one problem at a time? Are there times when we might want to solve multiple problems simultaneously? (is it always optimal to compartmentalize / solve one problem at a time? Are there times when we might want to solve multiple problems simultaneously?)

thinking about this in relation to dynamic programming – I think it serves as a good metaphor for why it can be useful to think about the structure of the problems you’re solving, and if there’s any overlap / reuse that can occur….seems that this is also related to decomposition / modularity